Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #28 - Allowing CatalogMetadata Serialisation/Deserialisation to/from JSON #169

Merged
merged 20 commits into from
Sep 13, 2021

Conversation

robons
Copy link
Contributor

@robons robons commented Sep 10, 2021

  • Added functionality to read/write dataclasses to/from JSON.
  • Added functinoality to override specific properties on said dataclasses (useful for overriding CatalogMetadata with the scraper's metadata).
  • Added param to csvqb CLI to accept additonal catalog metadata JSON to override defaults from info.json file.

Part of Issue #28.

@robons robons requested a review from canwaf September 10, 2021 13:05
Copy link
Contributor

@canwaf canwaf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of my comments are important more just little responses to seeing some things I knew we needed/like seeing.

"Location of a JSON file containing the Catalog Metadata for this qube. "
"If present, this overrides any configuration found in the info.json."
),
type=click.Path(exists=True, path_type=Path, file_okay=True, dir_okay=False),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one for the documentation on the series of overrides.

  1. Catalog information from info.json
  2. Catalog information from scraper
  3. Catalog information from catalog-metadata.json
  4. Runtime changes

@@ -33,34 +34,9 @@
T = TypeVar("T", bound=object)


def _from_dict(cls: Type[T], d: dict) -> T:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found where all the lines where dropped. :D

json.dump(self.as_json_dict(), f, indent=4)

@classmethod
def from_json_file(cls, file_path: Path) -> "CatalogMetadata":
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌

@robons robons changed the title Issue #28 Issue #28 - Allowing CatalogMetadata Serialisation/Deserialisation to/from JSON Sep 13, 2021
@robons robons merged commit 7741921 into main Sep 13, 2021
@robons robons deleted the robons-28 branch September 13, 2021 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants