Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add the /manifest_template endpoint to return a string representation of the manifest template #19367

Merged
merged 3 commits into from
Nov 14, 2022
Merged

Add the /manifest_template endpoint to return a string representation of the manifest template #19367

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
75a7e09
return a string representation of the manifest template
brianjlai Nov 13, 2022
1122b54
get rid of extra template
brianjlai Nov 13, 2022
0726a7f
add a comment
brianjlai Nov 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 50 additions & 1 deletion airbyte-connector-builder-server/connector_builder/impl/default_api.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,56 @@

class DefaultApiImpl(DefaultApi):
async def get_manifest_template(self) -> str:
return "Hello World"
return """version: "0.1.0"

definitions:
selector:
extractor:
field_pointer: []
requester:
url_base: "https://example.com"
http_method: "GET"
authenticator:
type: BearerAuthenticator
api_token: "{{ config['api_key'] }}"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned. In the generator this is defined as /{{ config['api_key'] }} because we need to escape the template so that the template generator doesn't fill this with anything. We want the {{ }} to actually be present in the file. But for the server version, we don't need to escape anything

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One way that you could reduce the duplication here would be to have a shared python method which takes in arguments for the api_token value and the spec title value, and returns this string with those arguments inserted. Then this method could call it and pass in the api_token without the slash, and the generator could call it and pass in the api_token with the slash, etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm yeah that sounds like it could work, but I might table that type of change for the future, I don't think its necessarily critical to keep things perfectly in sync in V0. And we could have some thorny edges trying to read the YAML template from airbyte-integrations/connector-templates/source-configuration-based/source_{{snakeCase name}}/{{snakeCase name}}.yaml.hbs across python packages 😅

retriever:
record_selector:
$ref: "*ref(definitions.selector)"
paginator:
type: NoPagination
requester:
$ref: "*ref(definitions.requester)"
base_stream:
retriever:
$ref: "*ref(definitions.retriever)"
customers_stream:
$ref: "*ref(definitions.base_stream)"
$options:
name: "customers"
primary_key: "id"
path: "/example"

streams:
- "*ref(definitions.customers_stream)"

check:
stream_names:
- "customers"

spec:
documentation_url: https://docsurl.com
connection_specification:
title: Source Name Spec # 'TODO: Replace this with the name of your source.'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this used to be {{capitalCase name}} in the generator. But it could be misleading for a developer thinking it should be interpolated when its not supposed to be.

type: object
required:
- api_key
additionalProperties: true
properties:
# 'TODO: This schema defines the configuration required for the source. This usually involves metadata such as database and/or authentication information.':
api_key:
type: string
description: API Key
"""

async def list_streams(self, streams_list_request_body: StreamsListRequestBody = Body(None, description="")) -> StreamsListRead:
raise Exception("not yet implemented")
Expand Down