Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

non-jdbc source connectors: Update additional properties from beta/GA specs and schemas #15042

Merged
merged 7 commits into from
Jul 26, 2022
Merged
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ RUN pip install .
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.9
LABEL io.airbyte.version=0.1.10
LABEL io.airbyte.name=airbyte/source-amazon-ads
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@
"oauth_config_specification": {
"complete_oauth_output_specification": {
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"refresh_token": {
"type": "string",
Expand All @@ -91,7 +91,7 @@
},
"complete_oauth_server_input_specification": {
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"client_id": {
"type": "string"
Expand All @@ -103,7 +103,7 @@
},
"complete_oauth_server_output_specification": {
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"client_id": {
"type": "string",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,14 @@ def schema_extra(cls, schema: Dict[str, Any], model: Type["BaseModel"]) -> None:
schema.pop("description", None)
# Remove required section so any missing attribute from API wont break object validation.
schema.pop("required", None)
# According to https://github.com/airbytehq/airbyte/issues/14196 set additionalProperties to True
if schema.pop("additionalProperties", None):
schema["additionalProperties"] = True
for name, prop in schema.get("properties", {}).items():
prop.pop("title", None)
prop.pop("description", None)
if prop.pop("additionalProperties", None):
prop["additionalProperties"] = True
allow_none = model.__fields__[name].allow_none
# Pydantic doesnt treat Union[None, Any] type correctly when
# generation jsonschema so we cant set field as nullable (i.e.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -100,23 +100,23 @@ advanced_auth:
oauth_config_specification:
complete_oauth_output_specification:
type: object
additionalProperties: false
additionalProperties: true
properties:
refresh_token:
type: string
path_in_connector_config:
- refresh_token
complete_oauth_server_input_specification:
type: object
additionalProperties: false
additionalProperties: true
properties:
client_id:
type: string
client_secret:
type: string
complete_oauth_server_output_specification:
type: object
additionalProperties: false
additionalProperties: true
properties:
client_id:
type: string
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ COPY main.py ./
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.23
LABEL io.airbyte.version=0.1.24
LABEL io.airbyte.name=airbyte/source-google-analytics-v4
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"name": "website_overview",
"json_schema": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"ga_date": {
"type": ["string"]
Expand Down Expand Up @@ -60,7 +60,7 @@
"name": "traffic_sources",
"json_schema": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"ga_date": {
"type": ["string"]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ def get_json_schema(self) -> Mapping[str, Any]:
schema: Dict[str, Any] = {
"$schema": "http://json-schema.org/draft-07/schema#",
"type": ["null", "object"],
"additionalProperties": False,
"additionalProperties": True,
"properties": {
"view_id": {"type": ["string"]},
},
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-notion/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,5 @@ COPY source_notion ./source_notion
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.6
LABEL io.airbyte.version=0.1.7
LABEL io.airbyte.name=airbyte/source-notion
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"title": { "type": "string" }
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
},
"external": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"url": {
"type": "string"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"color": { "type": "string" },
"text": { "type": "array", "items": { "$ref": "rich_text.json" } }
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"id": {
"type": "string"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"type": {
"enum": ["database_id", "page_id", "workspace"]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@
},
"text": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"content": {
"type": ["null", "string"]
},
"link": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"type": {
"enum": ["url"]
Expand All @@ -28,14 +28,14 @@
},
"rich_text": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"content": {
"type": ["null", "string"]
},
"link": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"type": {
"enum": ["url"]
Expand All @@ -49,7 +49,7 @@
},
"annotations": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"bold": {
"type": ["null", "boolean"]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"color": { "type": "string" },
"text": { "type": "array", "items": { "$ref": "rich_text.json" } },
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"type": ["null", "array"],
"items": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"type": {
"type": ["null", "string"]
Expand All @@ -20,7 +20,7 @@
},
"link": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"type": {
"enum": ["url"]
Expand All @@ -34,7 +34,7 @@
},
"annotations": {
"type": ["null", "object"],
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"bold": {
"type": ["null", "boolean"]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"additionalProperties": false,
"additionalProperties": true,
"properties": {
"object": {
"enum": ["user"]
Expand Down
21 changes: 11 additions & 10 deletions docs/integrations/sources/amazon-ads.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,13 +90,14 @@ Information about expected report generation waiting time you may find [here](ht

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:-----------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------|
| `0.1.9` | 2022-05-08 | [\#12541](https://github.com/airbytehq/airbyte/pull/12541) | Improve documentation for Beta |
| `0.1.8` | 2022-05-04 | [\#12482](https://github.com/airbytehq/airbyte/pull/12482) | Update input configuration copy |
| `0.1.7` | 2022-04-27 | [\#11730](https://github.com/airbytehq/airbyte/pull/11730) | Update fields in source-connectors specifications |
| `0.1.6` | 2022-04-20 | [\#11659](https://github.com/airbytehq/airbyte/pull/11659) | Add adId to products report |
| `0.1.5` | 2022-04-08 | [\#11430](https://github.com/airbytehq/airbyte/pull/11430) | `Added support OAuth2.0` |
| `0.1.4` | 2022-02-21 | [\#10513](https://github.com/airbytehq/airbyte/pull/10513) | `Increasing REPORT_WAIT_TIMEOUT for supporting report generation which takes longer time ` |
| `0.1.3` | 2021-12-28 | [\#8388](https://github.com/airbytehq/airbyte/pull/8388) | `Add retry if recoverable error occured for reporting stream processing` |
| `0.1.2` | 2021-10-01 | [\#6367](https://github.com/airbytehq/airbyte/pull/6461) | `Add option to pull data for different regions. Add option to choose profiles we want to pull data. Add lookback` |
| `0.1.1` | 2021-09-22 | [\#6367](https://github.com/airbytehq/airbyte/pull/6367) | `Add seller and vendor filters to profiles stream` |
| `0.1.0` | 2021-08-13 | [\#5023](https://github.com/airbytehq/airbyte/pull/5023) | `Initial version` |
| 0.1.10 | 2022-07-26 | [15042](https://github.com/airbytehq/airbyte/pull/15042) | Update `additionalProperties` field to true from schemas |
| 0.1.9 | 2022-05-08 | [12541](https://github.com/airbytehq/airbyte/pull/12541) | Improve documentation for Beta |
| 0.1.8 | 2022-05-04 | [12482](https://github.com/airbytehq/airbyte/pull/12482) | Update input configuration copy |
| 0.1.7 | 2022-04-27 | [11730](https://github.com/airbytehq/airbyte/pull/11730) | Update fields in source-connectors specifications |
| 0.1.6 | 2022-04-20 | [11659](https://github.com/airbytehq/airbyte/pull/11659) | Add adId to products report |
| 0.1.5 | 2022-04-08 | [11430](https://github.com/airbytehq/airbyte/pull/11430) | Added support OAuth2.0 |
| 0.1.4 | 2022-02-21 | [10513](https://github.com/airbytehq/airbyte/pull/10513) | Increasing REPORT_WAIT_TIMEOUT for supporting report generation which takes longer time |
| 0.1.3 | 2021-12-28 | [8388](https://github.com/airbytehq/airbyte/pull/8388) | Add retry if recoverable error occured for reporting stream processing |
| 0.1.2 | 2021-10-01 | [6367](https://github.com/airbytehq/airbyte/pull/6461) | Add option to pull data for different regions. Add option to choose profiles we want to pull data. Add lookback |
| 0.1.1 | 2021-09-22 | [6367](https://github.com/airbytehq/airbyte/pull/6367) | Add seller and vendor filters to profiles stream |
| 0.1.0 | 2021-08-13 | [5023](https://github.com/airbytehq/airbyte/pull/5023) | Initial version |
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,8 @@ Incremental sync is supported only if you add `ga:date` dimension to your custom

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:---------------------------------------------------------------------------------------------|
| 0.1.23 | 2022-07-22 | [00000](https://github.com/airbytehq/airbyte/pull/00000) | Add handle request daily quota error |
| 0.1.24 | 2022-07-26 | [15042](https://github.com/airbytehq/airbyte/pull/15042) | Update `additionalProperties` field to true from schemas |
| 0.1.23 | 2022-07-22 | [14949](https://github.com/airbytehq/airbyte/pull/14949) | Add handle request daily quota error |
| 0.1.22 | 2022-06-30 | [14298](https://github.com/airbytehq/airbyte/pull/14298) | Specify integer type for ga:dateHourMinute dimension |
| 0.1.21 | 2022-04-30 | [12500](https://github.com/airbytehq/airbyte/pull/12500) | Improve input configuration copy |
| 0.1.20 | 2022-04-28 | [12426](https://github.com/airbytehq/airbyte/pull/12426) | Expose `isDataGOlden` field and always resync data two days back to make sure it is golden |
Expand Down
1 change: 1 addition & 0 deletions docs/integrations/sources/notion.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ The Notion connector should not run into Notion API limitations under normal usa

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:----------------------------------------------------------|
| 0.1.7 | 2022-07-26 | [15042](https://github.com/airbytehq/airbyte/pull/15042) | Update `additionalProperties` field to true from shared schemas |
| 0.1.6 | 2022-07-21 | [14924](https://github.com/airbytehq/airbyte/pull/14924) | Remove `additionalProperties` field from schemas and spec |
| 0.1.5 | 2022-07-14 | [14706](https://github.com/airbytehq/airbyte/pull/14706) | Added OAuth2.0 authentication |
| 0.1.4 | 2022-07-07 | [14505](https://github.com/airbytehq/airbyte/pull/14505) | Fixed bug when normalization didn't run through |
Expand Down