You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some APIs like Facebook or Shopify output numbers as strings e.g: #4827 and #4841 This probably happens in more APIs as well.
Given that this is probably a ubiquitous issue we should have a simple and easy way to handle it.
There are some options here:
In the catalog type definition for such fields, add an annotation e.g: airbyte_type_hint: number indicating that a particular string is actually a number and then have Airbyte or the destination worker. This is nice because the source doesn't have to worry about transformation. On the other hand it is not ideal because we'd be relying on each destination to support this custom type hint.
Do the transformation in the source directly (e.g: hardcode which fields have this issue and transform in the code), making sure the JSON Schema type is output as number in the catalog. if we go this route we might want to add this capability into the CDK.
Something else I'm not thinking of
Describe the solution you’d like
Research and decide on which option to use
create follow up tickets for implementation
The text was updated successfully, but these errors were encountered:
Tell us about the problem you're trying to solve
Some APIs like Facebook or Shopify output numbers as strings e.g: #4827 and #4841 This probably happens in more APIs as well.
Given that this is probably a ubiquitous issue we should have a simple and easy way to handle it.
There are some options here:
airbyte_type_hint: number
indicating that a particular string is actually a number and then have Airbyte or the destination worker. This is nice because the source doesn't have to worry about transformation. On the other hand it is not ideal because we'd be relying on each destination to support this custom type hint.number
in the catalog. if we go this route we might want to add this capability into the CDK.Describe the solution you’d like
The text was updated successfully, but these errors were encountered: