#695 Move Code List SPARQL Queries to CodeListInspector #725

nimshi89 · 2023-02-06T10:35:01Z

transferred the following queries to cached properties:

select_codelist_csv_url.sparql (code_list_inspector.py) - reworked to link between the csv_url and conceptSchemeUrl
select_codelist_cols_by_csv_url.sparql (csvw_state.py) - replaced with a generic columns query in csvw_state

Adds a get_catalog_metadata API function to code_list_inspector.py which uses the results from the select_codelist_csv_url.sparql cached property.

Adds unit test coverage for new functions/properties

Class has been renamed to CodeListInspector

github-actions · 2023-02-07T13:10:42Z

ubuntu-latest-python3.9 test results

460 tests +4 460 ✔️ +4 2m 59s ⏱️ -16s
    8 suites ±0     0 💤 ±0
    8 files ±0     0 ❌ ±0

Results for commit 2e5142e. ± Comparison against base commit 9bea1bd.

This pull request removes 1 and adds 5 tests. Note that renamed tests count towards both.

tests.unit.utils.sparqlhandler.test_sparqlquerymanager ‑ test_select_qb_csv_url

tests.unit.cli.inspect.test_code_list_inspector ‑ test_code_list_table_identifiers
tests.unit.cli.inspect.test_code_list_inspector ‑ test_code_list_table_identifiers_error
tests.unit.cli.inspect.test_code_list_inspector ‑ test_get_catalog_metadata_for_concept_scheme
tests.unit.cli.inspect.test_code_list_inspector ‑ test_get_catalog_metadata_for_concept_scheme_error
tests.unit.cli.inspect.test_code_list_inspector ‑ test_get_table_identifiers_for_concept_scheme_error

♻️ This comment has been updated with latest results.

github-actions · 2023-02-07T13:11:38Z

ubuntu-latest-python3.11 test results

460 tests +4 460 ✔️ +4 2m 49s ⏱️ -15s
    8 suites ±0     0 💤 ±0
    8 files ±0     0 ❌ ±0

Results for commit 2e5142e. ± Comparison against base commit 9bea1bd.

This pull request removes 1 and adds 5 tests. Note that renamed tests count towards both.

tests.unit.utils.sparqlhandler.test_sparqlquerymanager ‑ test_select_qb_csv_url

tests.unit.cli.inspect.test_code_list_inspector ‑ test_code_list_table_identifiers
tests.unit.cli.inspect.test_code_list_inspector ‑ test_code_list_table_identifiers_error
tests.unit.cli.inspect.test_code_list_inspector ‑ test_get_catalog_metadata_for_concept_scheme
tests.unit.cli.inspect.test_code_list_inspector ‑ test_get_catalog_metadata_for_concept_scheme_error
tests.unit.cli.inspect.test_code_list_inspector ‑ test_get_table_identifiers_for_concept_scheme_error

♻️ This comment has been updated with latest results.

github-actions · 2023-02-07T13:23:32Z

windows-latest-python3.11 test results

    9 files ±0   10 suites ±0 7m 53s ⏱️ +17s
474 tests +4 474 ✔️ +4 0 💤 ±0 0 ❌ ±0
487 runs +4 487 ✔️ +4 0 💤 ±0 0 ❌ ±0

Results for commit 2e5142e. ± Comparison against base commit 9bea1bd.

This pull request removes 1 and adds 5 tests. Note that renamed tests count towards both.

tests.unit.utils.sparqlhandler.test_sparqlquerymanager ‑ test_select_qb_csv_url

tests.unit.cli.inspect.test_code_list_inspector ‑ test_code_list_table_identifiers
tests.unit.cli.inspect.test_code_list_inspector ‑ test_code_list_table_identifiers_error
tests.unit.cli.inspect.test_code_list_inspector ‑ test_get_catalog_metadata_for_concept_scheme
tests.unit.cli.inspect.test_code_list_inspector ‑ test_get_catalog_metadata_for_concept_scheme_error
tests.unit.cli.inspect.test_code_list_inspector ‑ test_get_table_identifiers_for_concept_scheme_error

♻️ This comment has been updated with latest results.

src/csvcubed/utils/sparql_handler/code_list_inspector.py

src/csvcubed/utils/sparql_handler/data_cube_state.py

tests/unit/cli/inspect/test_code_list_inspector.py

tests/unit/utils/sparqlhandler/test_code_list_state.py

NickPapONS

Looks good, makes sense to me! There are a few differences in the way cached properties are implemented compared to how we did it in the other tickets with myself and Sarah (data_cube_state or csvw_state), but I think these cached properties mostly work independently between code lists and data cubes. There will be quite a bit of conflict resolution needed when we merge all these 3 cached properties tickets together, but that's a natural part of splitting up the task the way we did. I don't see anything here that I think we wouldn't want to keep anyway.

src/csvcubed/cli/inspect/metadataprinter.py

src/csvcubed/utils/sparql_handler/code_list_inspector.py

src/csvcubed/utils/sparql_handler/csvw_state.py

src/csvcubed/utils/sparql_handler/data_cube_state.py

src/csvcubed/utils/sparql_handler/code_list_inspector.py

src/csvcubed/utils/sparql_handler/csvw_state.py

tests/behaviour/steps/inspectcli.py

tests/test-cases/cli/inspect/itis-industry-no-skos-inscheme.csv-metadata.json

tests/test-cases/cli/inspect/itis-industry.csv-metadata.json

tests/unit/cli/inspect/test_code_list_inspector.py

robons

Almost there, just tidy up those comments and I think we're pretty much there.

src/csvcubed/utils/sparql_handler/code_list_inspector.py

src/csvcubed/utils/sparql_handler/csvw_state.py

src/csvcubed/utils/sparql_handler/data_cube_state.py

tests/helpers/inspectors_cache.py

sonarcloud · 2023-02-15T14:57:55Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

0.0% Coverage
0.0% Duplication

robons · 2023-02-16T10:17:24Z

src/csvcubed/utils/sparql_handler/csvw_state.py

+    def get_column_definitions_for_csv(self, csv_url: str) -> List[ColumnDefinition]:
+        """
+        Returns the `ColumnDefinition`s for a given csv file, raises a KeyError if the csv_url
+        is not associated with a ColumnDefinition.


A list of column definitions.

nimshi89 and others added 11 commits February 2, 2023 09:02

f

5233d85

f

d67469c

f

5cd9941

f

3f0da00

f

61631c8

issue

a2e3efc

1 KeyError unit test fixed

66b992e

unit tests

93dba56

f

93ac8a7

fix

05bac38

ups

c4790d2

fix

88bece8

CharlesRendle reviewed Feb 7, 2023

View reviewed changes

nimshi89 added 3 commits February 8, 2023 09:14

fix

492647d

f

25d16a8

f

353c5fb

nimshi89 marked this pull request as ready for review February 8, 2023 10:59

NickPapONS reviewed Feb 10, 2023

View reviewed changes

robons changed the title ~~695 populate code list~~ #695 Move Code List SPARQL Queries to CodeListInspector Feb 13, 2023

robons suggested changes Feb 13, 2023

View reviewed changes

nimshi89 and others added 7 commits February 13, 2023 14:02

f

b2835bd

f

2168f99

f

53c2b93

f

3c52106

Merge remote-tracking branch 'origin/main' into 695_populate_code_list

0dc8a45

get_code_list_inpector() function + ammendments

fcdb967

import cleanup

4c6636f

CharlesRendle requested a review from robons February 14, 2023 15:33

robons reviewed Feb 15, 2023

View reviewed changes

f

2e5142e

CharlesRendle requested a review from robons February 15, 2023 17:48

robons approved these changes Feb 16, 2023

View reviewed changes

robons merged commit 2361d5d into main Feb 16, 2023

nimshi89 deleted the 695_populate_code_list branch February 24, 2023 08:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

#695 Move Code List SPARQL Queries to CodeListInspector #725

#695 Move Code List SPARQL Queries to CodeListInspector #725

nimshi89 commented Feb 6, 2023 •

edited

Loading

github-actions bot commented Feb 7, 2023 •

edited

Loading

github-actions bot commented Feb 7, 2023 •

edited

Loading

github-actions bot commented Feb 7, 2023 •

edited

Loading

NickPapONS left a comment

robons left a comment

sonarcloud bot commented Feb 15, 2023

robons Feb 16, 2023

#695 Move Code List SPARQL Queries to CodeListInspector #725

#695 Move Code List SPARQL Queries to CodeListInspector #725

Conversation

nimshi89 commented Feb 6, 2023 • edited Loading

github-actions bot commented Feb 7, 2023 • edited Loading

ubuntu-latest-python3.9 test results

github-actions bot commented Feb 7, 2023 • edited Loading

ubuntu-latest-python3.11 test results

github-actions bot commented Feb 7, 2023 • edited Loading

windows-latest-python3.11 test results

NickPapONS left a comment

Choose a reason for hiding this comment

robons left a comment

Choose a reason for hiding this comment

sonarcloud bot commented Feb 15, 2023

robons Feb 16, 2023

Choose a reason for hiding this comment

nimshi89 commented Feb 6, 2023 •

edited

Loading

github-actions bot commented Feb 7, 2023 •

edited

Loading

github-actions bot commented Feb 7, 2023 •

edited

Loading

github-actions bot commented Feb 7, 2023 •

edited

Loading