Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added static typing to data_readers/base_data.py and data_readers/json_data.py #666

Merged
merged 12 commits into from
Oct 7, 2022
Merged

Added static typing to data_readers/base_data.py and data_readers/json_data.py #666

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
3f85a5c
added type annotations to avro_data.py
Sanketh7 Sep 20, 2022
b3e5ebc
added missing return type
Sanketh7 Sep 20, 2022
e7c7cc8
removed extra import
Sanketh7 Sep 20, 2022
61fd535
Merge branch 'main' into data_readers_static_typing1
Sanketh7 Sep 20, 2022
18c015f
changed assertions to if statements
Sanketh7 Sep 26, 2022
fc9acc5
added static typing to base_data and json_data
Sanketh7 Sep 27, 2022
e4fe85e
Merge branch 'main' into data_readers_static_typing2
Sanketh7 Sep 30, 2022
f276b34
fixed issue with data being multiple types
Sanketh7 Oct 4, 2022
318129e
changed call in reload back to self.__init__ with type: ignore
Sanketh7 Oct 4, 2022
af9436a
changed isinstance to cast in avro_data
Sanketh7 Oct 4, 2022
11a5b6d
Merge branch 'main' into data_readers_static_typing2
Sanketh7 Oct 6, 2022
a842db4
Merge branch 'main' into data_readers_static_typing2
taylorfturner Oct 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions dataprofiler/data_readers/avro_data.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""Contains class for saving and loading spreadsheet data."""
from io import BytesIO, StringIO
from typing import Any, Dict, List, Optional, Union
from typing import Any, Dict, List, Optional, Union, cast

import fastavro

Expand Down Expand Up @@ -91,13 +91,19 @@ def is_match(
options = dict()

# get current position of stream
if data_utils.is_stream_buffer(file_path) and not isinstance(file_path, str):
if data_utils.is_stream_buffer(file_path):
Sanketh7 marked this conversation as resolved.
Show resolved Hide resolved
file_path = cast(
Union[StringIO, BytesIO], file_path
) # guaranteed by is_stream_buffer
starting_location = file_path.tell()

is_valid_avro = fastavro.is_avro(file_path)

# return to original position in stream
if data_utils.is_stream_buffer(file_path) and not isinstance(file_path, str):
if data_utils.is_stream_buffer(file_path):
file_path = cast(
Union[StringIO, BytesIO], file_path
) # guaranteed by is_stream_buffer
file_path.seek(starting_location, 0)

return is_valid_avro
Expand Down