Skip to content

Commit

Permalink
Fixing custom S3 endpoint read fail
Browse files Browse the repository at this point in the history
Signed-off-by: llFireHawkll <[email protected]>
  • Loading branch information
llFireHawkll committed Jun 13, 2022
1 parent 452dcd3 commit d5f1da1
Showing 1 changed file with 9 additions and 3 deletions.
12 changes: 9 additions & 3 deletions sdk/python/feast/infra/offline_stores/file_source.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,9 +179,15 @@ def get_table_column_names_and_types(
filesystem, path = FileSource.create_filesystem_and_path(
self.path, self.file_options.s3_endpoint_override
)
schema = ParquetDataset(
path if filesystem is None else filesystem.open_input_file(path)
).schema.to_arrow_schema()
# Adding support for different file format path
# based on S3 filesystem
if filesystem is None:
schema = ParquetDataset(path).schema.to_arrow_schema()
else:
schema = ParquetDataset(
filesystem.open_input_file(path), filesystem=filesystem
).schema

return zip(schema.names, map(str, schema.types))

@staticmethod
Expand Down

0 comments on commit d5f1da1

Please sign in to comment.