Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

non-jdbc source connectors: Update additional properties from beta/GA specs and schemas #15042

Merged
merged 7 commits into from
Jul 26, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
- name: Amazon Ads
sourceDefinitionId: c6b0a29e-1da9-4512-9002-7bfd0cba2246
dockerRepository: airbyte/source-amazon-ads
dockerImageTag: 0.1.9
dockerImageTag: 0.1.10
documentationUrl: https://docs.airbyte.io/integrations/sources/amazon-ads
icon: amazonads.svg
sourceType: api
Expand Down Expand Up @@ -327,7 +327,7 @@
- name: Google Analytics
sourceDefinitionId: eff3616a-f9c3-11eb-9a03-0242ac130003
dockerRepository: airbyte/source-google-analytics-v4
dockerImageTag: 0.1.23
dockerImageTag: 0.1.24
documentationUrl: https://docs.airbyte.io/integrations/sources/google-analytics-v4
icon: google-analytics.svg
sourceType: api
Expand Down Expand Up @@ -612,7 +612,7 @@
- name: Notion
sourceDefinitionId: 6e00b415-b02e-4160-bf02-58176a0ae687
dockerRepository: airbyte/source-notion
dockerImageTag: 0.1.6
dockerImageTag: 0.1.7
documentationUrl: https://docs.airbyte.io/integrations/sources/notion
icon: notion.svg
sourceType: api
Expand Down
12 changes: 6 additions & 6 deletions airbyte-config/init/src/main/resources/seed/source_specs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-amazon-ads:0.1.9"
- dockerImage: "airbyte/source-amazon-ads:0.1.10"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/amazon-ads"
connectionSpecification:
Expand Down Expand Up @@ -187,23 +187,23 @@
oauth_config_specification:
complete_oauth_output_specification:
type: "object"
additionalProperties: false
additionalProperties: true
properties:
refresh_token:
type: "string"
path_in_connector_config:
- "refresh_token"
complete_oauth_server_input_specification:
type: "object"
additionalProperties: false
additionalProperties: true
properties:
client_id:
type: "string"
client_secret:
type: "string"
complete_oauth_server_output_specification:
type: "object"
additionalProperties: false
additionalProperties: true
properties:
client_id:
type: "string"
Expand Down Expand Up @@ -2937,7 +2937,7 @@
oauthFlowOutputParameters:
- - "access_token"
- - "refresh_token"
- dockerImage: "airbyte/source-google-analytics-v4:0.1.23"
- dockerImage: "airbyte/source-google-analytics-v4:0.1.24"
spec:
documentationUrl: "https://docs.airbyte.io/integrations/sources/google-analytics-v4"
connectionSpecification:
Expand Down Expand Up @@ -5930,7 +5930,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-notion:0.1.6"
- dockerImage: "airbyte/source-notion:0.1.7"
spec:
documentationUrl: "https://docs.airbyte.io/integrations/sources/notion"
connectionSpecification:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ RUN pip install .
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.9
LABEL io.airbyte.version=0.1.10
LABEL io.airbyte.name=airbyte/source-amazon-ads
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@
"oauth_config_specification": {
"complete_oauth_output_specification": {
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"refresh_token": {
"type": "string",
Expand All @@ -91,7 +91,7 @@
},
"complete_oauth_server_input_specification": {
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"client_id": {
"type": "string"
Expand All @@ -103,7 +103,7 @@
},
"complete_oauth_server_output_specification": {
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"client_id": {
"type": "string",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,14 @@ def schema_extra(cls, schema: Dict[str, Any], model: Type["BaseModel"]) -> None:
schema.pop("description", None)
# Remove required section so any missing attribute from API wont break object validation.
schema.pop("required", None)
# According to https://github.com/airbytehq/airbyte/issues/14196 set additionalProperties to True
if schema.pop("additionalProperties", None):
schema["additionalProperties"] = True
for name, prop in schema.get("properties", {}).items():
prop.pop("title", None)
prop.pop("description", None)
if prop.pop("additionalProperties", None):
prop["additionalProperties"] = True
allow_none = model.__fields__[name].allow_none
# Pydantic doesnt treat Union[None, Any] type correctly when
# generation jsonschema so we cant set field as nullable (i.e.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -100,23 +100,23 @@ advanced_auth:
oauth_config_specification:
complete_oauth_output_specification:
type: object
additionalProperties: false
additionalProperties: true
properties:
refresh_token:
type: string
path_in_connector_config:
- refresh_token
complete_oauth_server_input_specification:
type: object
additionalProperties: false
additionalProperties: true
properties:
client_id:
type: string
client_secret:
type: string
complete_oauth_server_output_specification:
type: object
additionalProperties: false
additionalProperties: true
properties:
client_id:
type: string
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ COPY main.py ./
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.23
LABEL io.airbyte.version=0.1.24
LABEL io.airbyte.name=airbyte/source-google-analytics-v4
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"name": "website_overview",
"json_schema": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"ga_date": {
"type": ["string"]
Expand Down Expand Up @@ -60,7 +60,7 @@
"name": "traffic_sources",
"json_schema": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"ga_date": {
"type": ["string"]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ def get_json_schema(self) -> Mapping[str, Any]:
schema: Dict[str, Any] = {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": ["null", "object"],
"additionalProperties": False,
"additionalProperties": True,
"properties": {
"view_id": {"type": ["string"]},
},
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-notion/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,5 @@ COPY source_notion ./source_notion
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.6
LABEL io.airbyte.version=0.1.7
LABEL io.airbyte.name=airbyte/source-notion
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"title": { "type": "string" }
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
},
"external": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"url": {
"type": "string"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"color": { "type": "string" },
"text": { "type": "array", "items": { "$ref": "rich_text.json" } }
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"id": {
"type": "string"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"type": {
"enum": ["database_id", "page_id", "workspace"]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@
},
"text": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"content": {
"type": ["null", "string"]
},
"link": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"type": {
"enum": ["url"]
Expand All @@ -28,14 +28,14 @@
},
"rich_text": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"content": {
"type": ["null", "string"]
},
"link": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"type": {
"enum": ["url"]
Expand All @@ -49,7 +49,7 @@
},
"annotations": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"bold": {
"type": ["null", "boolean"]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"color": { "type": "string" },
"text": { "type": "array", "items": { "$ref": "rich_text.json" } },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"type": ["null", "array"],
"items": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"type": {
"type": ["null", "string"]
Expand All @@ -20,7 +20,7 @@
},
"link": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"type": {
"enum": ["url"]
Expand All @@ -34,7 +34,7 @@
},
"annotations": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"bold": {
"type": ["null", "boolean"]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"object": {
"enum": ["user"]
Expand Down
21 changes: 11 additions & 10 deletions docs/integrations/sources/amazon-ads.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,13 +90,14 @@ Information about expected report generation waiting time you may find [here](ht

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:-----------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------|
| `0.1.9` | 2022-05-08 | [\#12541](https://github.com/airbytehq/airbyte/pull/12541) | Improve documentation for Beta |
| `0.1.8` | 2022-05-04 | [\#12482](https://github.com/airbytehq/airbyte/pull/12482) | Update input configuration copy |
| `0.1.7` | 2022-04-27 | [\#11730](https://github.com/airbytehq/airbyte/pull/11730) | Update fields in source-connectors specifications |
| `0.1.6` | 2022-04-20 | [\#11659](https://github.com/airbytehq/airbyte/pull/11659) | Add adId to products report |
| `0.1.5` | 2022-04-08 | [\#11430](https://github.com/airbytehq/airbyte/pull/11430) | `Added support OAuth2.0` |
| `0.1.4` | 2022-02-21 | [\#10513](https://github.com/airbytehq/airbyte/pull/10513) | `Increasing REPORT_WAIT_TIMEOUT for supporting report generation which takes longer time ` |
| `0.1.3` | 2021-12-28 | [\#8388](https://github.com/airbytehq/airbyte/pull/8388) | `Add retry if recoverable error occured for reporting stream processing` |
| `0.1.2` | 2021-10-01 | [\#6367](https://github.com/airbytehq/airbyte/pull/6461) | `Add option to pull data for different regions. Add option to choose profiles we want to pull data. Add lookback` |
| `0.1.1` | 2021-09-22 | [\#6367](https://github.com/airbytehq/airbyte/pull/6367) | `Add seller and vendor filters to profiles stream` |
| `0.1.0` | 2021-08-13 | [\#5023](https://github.com/airbytehq/airbyte/pull/5023) | `Initial version` |
| 0.1.10 | 2022-07-26 | [15042](https://github.com/airbytehq/airbyte/pull/15042) | Update `additionalProperties` field to true from schemas |
| 0.1.9 | 2022-05-08 | [12541](https://github.com/airbytehq/airbyte/pull/12541) | Improve documentation for Beta |
| 0.1.8 | 2022-05-04 | [12482](https://github.com/airbytehq/airbyte/pull/12482) | Update input configuration copy |
| 0.1.7 | 2022-04-27 | [11730](https://github.com/airbytehq/airbyte/pull/11730) | Update fields in source-connectors specifications |
| 0.1.6 | 2022-04-20 | [11659](https://github.com/airbytehq/airbyte/pull/11659) | Add adId to products report |
| 0.1.5 | 2022-04-08 | [11430](https://github.com/airbytehq/airbyte/pull/11430) | Added support OAuth2.0 |
| 0.1.4 | 2022-02-21 | [10513](https://github.com/airbytehq/airbyte/pull/10513) | Increasing REPORT_WAIT_TIMEOUT for supporting report generation which takes longer time |
| 0.1.3 | 2021-12-28 | [8388](https://github.com/airbytehq/airbyte/pull/8388) | Add retry if recoverable error occured for reporting stream processing |
| 0.1.2 | 2021-10-01 | [6367](https://github.com/airbytehq/airbyte/pull/6461) | Add option to pull data for different regions. Add option to choose profiles we want to pull data. Add lookback |
| 0.1.1 | 2021-09-22 | [6367](https://github.com/airbytehq/airbyte/pull/6367) | Add seller and vendor filters to profiles stream |
| 0.1.0 | 2021-08-13 | [5023](https://github.com/airbytehq/airbyte/pull/5023) | Initial version |
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,8 @@ Incremental sync is supported only if you add `ga:date` dimension to your custom

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:---------------------------------------------------------------------------------------------|
| 0.1.23 | 2022-07-22 | [00000](https://github.com/airbytehq/airbyte/pull/00000) | Add handle request daily quota error |
| 0.1.24 | 2022-07-26 | [15042](https://github.com/airbytehq/airbyte/pull/15042) | Update `additionalProperties` field to true from schemas |
| 0.1.23 | 2022-07-22 | [14949](https://github.com/airbytehq/airbyte/pull/14949) | Add handle request daily quota error |
| 0.1.22 | 2022-06-30 | [14298](https://github.com/airbytehq/airbyte/pull/14298) | Specify integer type for ga:dateHourMinute dimension |
| 0.1.21 | 2022-04-30 | [12500](https://github.com/airbytehq/airbyte/pull/12500) | Improve input configuration copy |
| 0.1.20 | 2022-04-28 | [12426](https://github.com/airbytehq/airbyte/pull/12426) | Expose `isDataGOlden` field and always resync data two days back to make sure it is golden |
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/notion.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ The Notion connector should not run into Notion API limitations under normal usa

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:----------------------------------------------------------|
| 0.1.7 | 2022-07-26 | [15042](https://github.com/airbytehq/airbyte/pull/15042) | Update `additionalProperties` field to true from shared schemas |
| 0.1.6 | 2022-07-21 | [14924](https://github.com/airbytehq/airbyte/pull/14924) | Remove `additionalProperties` field from schemas and spec |
| 0.1.5 | 2022-07-14 | [14706](https://github.com/airbytehq/airbyte/pull/14706) | Added OAuth2.0 authentication |
| 0.1.4 | 2022-07-07 | [14505](https://github.com/airbytehq/airbyte/pull/14505) | Fixed bug when normalization didn't run through |
Expand Down