Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 Source Linkedin Ads - for adDirectSponsoredContents stream skip accounts which are not part of organization #18111

Merged
merged 6 commits into from
Oct 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -566,7 +566,7 @@
- name: LinkedIn Ads
sourceDefinitionId: 137ece28-5434-455c-8f34-69dc3782f451
dockerRepository: airbyte/source-linkedin-ads
dockerImageTag: 0.1.11
dockerImageTag: 0.1.12
documentationUrl: https://docs.airbyte.com/integrations/sources/linkedin-ads
icon: linkedin.svg
sourceType: api
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5673,7 +5673,7 @@
path_in_connector_config:
- "credentials"
- "client_secret"
- dockerImage: "airbyte/source-linkedin-ads:0.1.11"
- dockerImage: "airbyte/source-linkedin-ads:0.1.12"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/linkedin-ads"
connectionSpecification:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,5 +33,5 @@ COPY source_linkedin_ads ./source_linkedin_ads
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.11
LABEL io.airbyte.version=0.1.12
LABEL io.airbyte.name=airbyte/source-linkedin-ads
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,24 @@ def request_params(self, stream_state: Mapping[str, Any], stream_slice: Mapping[
params["q"] = self.search_param
return params

def read_records(
self, stream_state: Mapping[str, Any] = None, stream_slice: Optional[Mapping[str, Any]] = None, **kwargs
) -> Iterable[Mapping[str, Any]]:
stream_state = stream_state or {}
parent_stream = self.parent_stream(config=self.config)
for record in parent_stream.read_records(**kwargs):

if record.get("reference", "").startswith("urn:li:person"):
self.logger.warn(
f'Skip {record.get("name")} account, ORGANIZATION permissions required, but referenced to PERSON {record.get("reference")}'
)
continue

child_stream_slice = super(LinkedInAdsStreamSlicing, self).read_records(
stream_slice=get_parent_stream_values(record, self.parent_values_map), **kwargs
)
yield from self.filter_records_newer_than_state(stream_state=stream_state, records_slice=child_stream_slice)


class LinkedInAdsAnalyticsStream(IncrementalLinkedinAdsStream):
"""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,19 @@ def test_request_headers(self):
assert result == expected


class TestAccountUsers:
stream: AccountUsers = AccountUsers(TEST_CONFIG)

def test_state_checkpoint_interval(self):
assert self.stream.state_checkpoint_interval == 500

def test_get_updated_state(self):
state = self.stream.get_updated_state(
current_stream_state={"lastModified": "2021-01-01"}, latest_record={"lastModified": "2021-08-01"}
)
assert state == {"lastModified": "2021-08-01"}


class TestLinkedInAdsStreamSlicing:
@pytest.mark.parametrize(
"stream_cls, slice, expected",
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/linkedin-ads.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,7 @@ After 5 unsuccessful attempts - the connector will stop the sync operation. In s

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------|
| 0.1.12 | 2022-10-18 | [18111](https://github.com/airbytehq/airbyte/pull/18111) | for adDirectSponsoredContents stream skip accounts which are part of organization |
| 0.1.11 | 2022-10-07 | [17724](https://github.com/airbytehq/airbyte/pull/17724) | Retry 429/5xx errors when refreshing access token |
| 0.1.10 | 2022-09-28 | [17326](https://github.com/airbytehq/airbyte/pull/17326) | Migrate to per-stream states. |
| 0.1.9 | 2022-07-21 | [14924](https://github.com/airbytehq/airbyte/pull/14924) | Remove `additionalProperties` field from schemas |
Expand Down