Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TG2-AMENDMENT_COUNTRYCODE_STANDARDIZED #48

Open
iDigBioBot opened this issue Jan 5, 2018 · 22 comments
Open

TG2-AMENDMENT_COUNTRYCODE_STANDARDIZED #48

iDigBioBot opened this issue Jan 5, 2018 · 22 comments
Labels
Amendment Conformance CORE TG2 CORE tests ISO/DCMI STANDARD SPACE Test Tests created by TG2, either CORE, Supplementary or DO NOT IMPLEMENT TG2 VOCABULARY

Comments

@iDigBioBot
Copy link
Collaborator

iDigBioBot commented Jan 5, 2018

TestField Value
GUID fec5ffe6-3958-4312-82d9-ebcca0efb350
Label AMENDMENT_COUNTRYCODE_STANDARDIZED
Description Proposes an amendment to the value of dwc:countryCode if it can be interpreted as an ISO 3166-1-alpha-2 country code.
TestType Amendment
Darwin Core Class dcterms:Location
Information Elements ActedUpon dwc:countryCode
Information Elements Consulted
Expected Response EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISTITES_NOT_MET if the value of dwc:countryCode is bdq:Empty; AMENDED the value of dwc:countryCode if it can be unambiguously interpreted to a value in the bdq:sourceAuthority; otherwise NOT_AMENDED
Data Quality Dimension Conformance
Term-Actions COUNTRYCODE_STANDARDIZED
Parameter(s)
Source Authority bdq:sourceAuthority default = "ISO 3166-1-alpha-2" {[https://www.iso.org/iso-3166-country-codes.html]} {ISO 3166-1-alpha-2 Country Code search [https://www.iso.org/obp/ui/#search]}
Specification Last Updated 2024-11-09
Examples [dwc:countryCode="Australia": Response.status=AMENDED, Response.result=dwc:countryCode="AU", Response.comment="dwc:countryCode contains an interpretable value"]
[dwc:countryCode="Aust.": Response.status=NOT_AMENDED, Response.result=, Response.comment="dwc:countryCode contains an ambiguous value"]
Source
References
Example Implementations (Mechanisms)
Link to Specification Source Code
Notes This test supports conformance with the recommendation in the comment on dwc:countryCode: "Recommended best practice is to use an ISO 3166-1-alpha-2 country code. " Three letter codes should be amended to the matching two letter code.
@iDigBioBot
Copy link
Collaborator Author

Comment by Lee Belbin (@Tasilee) migrated from spreadsheet:
Added post scoring for consistency

@ArthurChapman ArthurChapman added VOCABULARY Test Tests created by TG2, either CORE, Supplementary or DO NOT IMPLEMENT labels Jan 16, 2018
@tucotuco tucotuco added the Parameterized Test requires a parameter label Nov 5, 2018
@chicoreus
Copy link
Collaborator

Output Type is missing from the Field/Value table. @Tasilee please correct (looks like it should be Output Type = Amendment) and review table for any other errors.

@Tasilee
Copy link
Collaborator

Tasilee commented Aug 7, 2019

Thanks @chicoreus. Done.

@ArthurChapman
Copy link
Collaborator

ArthurChapman commented Sep 5, 2019

Reference and Parameters updated in accordance with #20. Removed Parameterized.

@ArthurChapman ArthurChapman removed the Parameterized Test requires a parameter label Sep 5, 2019
@tucotuco
Copy link
Member

tucotuco commented Sep 6, 2019

Wait! No!

None of those references allows you to make an amendment, they only provide the controlled vocabulary. I have been working on a general response in issue #178 all day, redoing it several times to try to be succinct. It will be forthcoming momentarily.

@Tasilee
Copy link
Collaborator

Tasilee commented Sep 8, 2019

@tucotuco: Given your comments on #178, I am presuming you are implying that we cannot AMEND without a thesaurus as in the example lookup "AU" -> "036" ?

In this case, we sort of do have something akin to a thesaurus in the References where we are implying that we code in the test something like "find AU among terms {...} and extract best matching ISO 3166-1-alpha-2 country code" ?

@tucotuco
Copy link
Member

tucotuco commented Sep 9, 2019

@Tasilee I don't think the pattern matching method in implementations is a good idea for reasons stated elsewhere. I still think a maintained lookup thesaurus is the way to go.

@Tasilee
Copy link
Collaborator

Tasilee commented Sep 9, 2019

@tucotuco - given your comments on #178 - I agree. Thanks John. I guess the process of converting me to the need for thesauri is something we can use in the paper and education generally.

@Tasilee
Copy link
Collaborator

Tasilee commented Mar 7, 2023

From recent discussions, I have changed the Expected Response from

EXTERNAL_PREREQUISITES_NOT_MET if the ISO 3166 service is not available; INTERNAL_PREREQUISTITES_NOT_MET if the value of dwc:countryCode is EMPTY; AMENDED the value of dwc:countryCode if it can be unambiguously interpreted from bdq:sourceAuthority; otherwise NOT_AMENDED

to

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISTITES_NOT_MET if the value of dwc:countryCode is EMPTY; AMENDED the value of dwc:countryCode if it can be unambiguously interpreted from bdq:sourceAuthority; otherwise NOT_AMENDED

@chicoreus
Copy link
Collaborator

This one shouldn't be parameterized. The source authority is also entangling the authority (per the Note on dwc:countryCode "Recommended best practice is to use an ISO 3166-1-alpha-2 country code.", ISO 3166-1-alpha-2) with places where the (non-free) authority may be available or consulted. Also, the endpoint specified, https://restcountries.eu /#api-endpoints-list-of-codes appears to have been unresponsive for some time.

Consider amending (with no parameter, and no source authority) to:

EXTERNAL_PREREQUISITES_NOT_MET if a configured service for checking ISO 3166 country codes is not available; INTERNAL_PREREQUISTITES_NOT_MET if the value of dwc:countryCode is EMPTY; AMENDED the value of dwc:countryCode if it can be unambiguously interpreted as a ISO 3166-1-alpha-2; otherwise NOT_AMENDED

In some places we take the position that for data to have quality for CORE purposes, it must conform with the recommendation in the non-normative Note/Comment of a Darwin Core term. This is a case where we should do that.

We also make no recommendations about the scope of values that an implementation should consider in assessing " unambiguously interpreted", these could include case insensitive matches, matches on ISO 3 letter codes, matches on ISO numeric codes, matches on country names, matches and conflicting matches imparting ambiguity on other codes.

@chicoreus
Copy link
Collaborator

Another interpretation of the parameter is that it implies the existence of one or more service that can take a string, and return a two letter ISO code that can be unambiguously interpreted for that string. Such a service doesn't exist.

chicoreus added a commit to FilteredPush/geo_ref_qc that referenced this issue Jun 29, 2023
…-06-28) specifications. Addressing tdwg/bdq#48 AMENDMENT_COUNTRYCODE_STANDARDIZED with implementation without a parameter and notes that issue needs work.  Adding more support for backing to lookup country codes along with unit tests.  Work in progress on tdwg/bdq#43 AMENDMENT_COORDINATES_CONVERTED not yet working implementation using org.geotools library.
chicoreus added a commit to FilteredPush/geo_ref_qc that referenced this issue Jun 29, 2023
…-06-28) specifications. Tuning tdwg/bdq#48 AMENDMENT_COUNTRYCODE_STANDARDIZED to pass DataID: 265, matching numeric counntry codes with non-standard leading zeroes.
@ArthurChapman
Copy link
Collaborator

Agreed a lot of what you say @chicoreus. We removed Parameterized back in September 2019. The citing of bdq:sourceAuthority does not mean it is Parameterized - there are two tests like that - the other is #20. If we change the wording here - we should do likewise in #20. I am happy to go with your wording above and remove reference to bdq:sourceAuthority

@Tasilee
Copy link
Collaborator

Tasilee commented Jun 29, 2023

I agree, but if the Expected response is self-contained, then there is no need for a bdq:sourceAuthority?

@ArthurChapman
Copy link
Collaborator

ArthurChapman commented Jun 29, 2023

As suggested in #20 - if agreed - can remove the reference to bdq:sourceAuthority in Source Authority - and it shouldn't have anything in Parameter(s) anyway - as it is not Parameterized.

@Tasilee
Copy link
Collaborator

Tasilee commented Jun 29, 2023

OK, go for it, but we need to capture "https://restcountries.eu/#api-endpoints-list-of-codes, https://www.iso.org/obp/ui/#search" in References, and given what @chicoreus said this morning, we should use a standard phrase to alert implementers...

API endpoint [https://restcountries.eu/#api-endpoints-list-of-codes, https://www.iso.org/obp/ui/#search]

?

@Tasilee
Copy link
Collaborator

Tasilee commented Jul 3, 2023

Removed "bdq:sourceAuthority" from Parameter(s) as there is only one: ISO 3166.

@Tasilee
Copy link
Collaborator

Tasilee commented Jul 4, 2023

Amended Source Authority value from

"bdq:sourceAuthority is "ISO 3166-1-alpha-2" [https://restcountries.eu/#api-endpoints-list-of-codes, https://www.iso.org/obp/ui/#search]"

to

"{bdq:sourceAuthority = ISO 3166-1-alpha-2} { Country codes [https://www.iso.org/obp/ui/#search]}"

to align with suggested syntax and better link.

@Tasilee
Copy link
Collaborator

Tasilee commented Jul 4, 2023

Amended Source Authority to align with @chicoreus suggested syntax

{bdq:sourceAuthority = ISO 3166-1-alpha-2} { Country codes [https://www.iso.org/obp/ui/#search]}

to

bdq:sourceAuthority default = "ISO 3166-1-alpha-2 country codes" { [https://www.iso.org/obp/ui/#search]}

@Tasilee Tasilee removed the NEEDS WORK label Jul 4, 2023
@Tasilee
Copy link
Collaborator

Tasilee commented Jul 11, 2023

Post Zoom 11/7/2023, I have aligned the Source Authority with the suggested syntax:

bdq:sourceAuthority default = "ISO 3166-1-alpha-2 country codes" {[https://www.iso.org/obp/ui/#search]}

to

bdq:sourceAuthority default = "ISO 3166 Country Codes" {[https://www.iso.org/iso-3166-country-codes.html]} {ISO 3166-1-alpha-2 Country Code search [https://www.iso.org/obp/ui/#search]}

@Tasilee
Copy link
Collaborator

Tasilee commented Sep 16, 2023

Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted". Also changed "Field" to "TestField" and "Output Type" to "TestType".

@chicoreus
Copy link
Collaborator

Source authority needs partial revision, it needs to be explicit that the authority is the two letter code, not any of the set of codes that comprise ISO 3166.

Changes to the string constant for source authorities must not be done lightly, implementations will depend on recognition of that exact string.

Changing the string constant in the source authority from "ISO 3166 Country Codes" back to the explicit "ISO 3166-1-alpha-2"

@chicoreus
Copy link
Collaborator

Edited the description to be consistent with that of VALIDATION_COUNTRYCODE_STANDARD.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Amendment Conformance CORE TG2 CORE tests ISO/DCMI STANDARD SPACE Test Tests created by TG2, either CORE, Supplementary or DO NOT IMPLEMENT TG2 VOCABULARY
Projects
None yet
Development

No branches or pull requests

5 participants