Merge pull request #66 from ehn-dcc-development/release/2.11.0

Release version 2.11.0

CHANGELOG.md

@@ -1,5 +1,18 @@
 # Change log
 
+## Release 2.11.0
+
+* Add a value set that encodes Annex A of the guidelines document, including its derivation.
+  See the [README](./annex-A/README.md) for more details.
+* W.r.t. the ["COVID-19 vaccine or prophylaxis" value set](./vaccine-prophylaxis.json), corresponding to the table in § 2.2 in the guidelines document:
+    1. The "J07BX03" ACT code has been deprecated by setting its `active` field's value to `false` and adding a postfix ` (deprecated)` to the `display` field's value.
+    2. The first two entries, for codes "1119349007" and "1119305005", have been swapped to match the order in the document's table.
+    3. Entries for SCT codes "28531000087107" (which replaces the "J07BX03" ACT code), "30141000087107", and "1187593009" have been added.
+* W.r.t. to the ["vaccine encoding instructions" value set](./vaccine-encoding-instructions.json), corresponding to Annex A of the guidelines document:
+    1. All occurrences of the "J07BX03" ACT code in , have been replaced by the "28531000087107" SCT code.
+    2. The derivation script for this value set has been updated to check for deprecated ACT/SCT codes.
+       (It also reports the correct row numbers now.)
+
 
 ## Release 2.10.0
 
@@ -13,7 +26,7 @@
 
   The [guidelines document v1.13 has been adopted and published](https://ec.europa.eu/health/sites/default/files/ehealth/docs/digital-green-value-sets_en.pdf) through the [eHealth network page on the EU DCC](https://ec.europa.eu/health/ehealth/covid-19_en).
 
-  Version 1.12 of this document has also been adopted after v1.11, but v1.12 only adds a textual clarification which has no impact on the (implementation of the) value sets themselves, and which has been removed again in v1.13.
+  Version 1.12 of this document has also been adopted after v1.11, but v1.12 only adds a textual clarification which has no impact on the (implementation of the) value sets themselves, and an Annex B which has been removed again in v1.13.
 
 
 ## Release 2.9.0

README.md

@@ -2,7 +2,7 @@
 
 This repository contains the value sets referenced by the EU Digital COVID Certificate (DCC) [JSON Schema](https://github.com/ehn-dcc-development/ehn-dcc-schema).
 
-Release: 2.10.0
+Release: 2.11.0
 
 Note that these value sets do not form a core part of the [JSON Schema](https://github.com/ehn-dcc-development/ehn-dcc-schema) but are referenced by it. These value sets can (and should) be regularly updated and distributed from the main Digital COVID Certificate Gateway (DCCG), under clear versioning and release management.
 

annex-A/README.md

@@ -0,0 +1,32 @@
+# Vaccine encoding instructions
+
+Annex A of the [document on “eHealth Network Guidelines on Value Sets
+for EU Digital COVID Certificates”](https://ec.europa.eu/health/sites/default/files/ehealth/docs/digital-green-value-sets_en.pdf)(eHN, DCC) contains a table that shows the coding combinations of the different vaccine medicinal products with their recommended ATC or SNOMED CT ‘Vaccine’ concept, along with their corresponding marketing authorization holder or manufacturer.
+The table is included to assist implementation efforts.
+
+Until release version 2.10.0 of the eHN EU DCC value sets the table in this Annex A was not encoded in a JSON format in this repository.
+Because of ongoing efforts relating to a WHO/G20 pilot, it has become advantageous to encode the table in Annex A.
+This has resulted in the [`vaccine-encoding-instructions.json` JSON file](../vaccine-encoding-instructions.json).
+
+To reduce tedious manual work (which is inherently somewhat error-prone), encoding the table to produce the corresponding JSON value set file has been (somewhat) automated.
+The steps to perform are:
+
+1. Perform a textual copy of the table from the guidelines document to the [`annex-A/table.tsv` file](./table.tsv).
+  This seems to work best from a `.docx` version of that document, rather than the published PDF file.
+2. Clean up the [`annex-A/table.tsv` file](./table.tsv) so that every line in it corresponds to exactly one row in the table.
+  It might be necessary to remove some newlines or other characters for that.
+3. Run the [`build.sh` build script](./build.sh).
+  This should produce the [`vaccine-encoding-instructions.json` JSON file](../vaccine-encoding-instructions.json).
+  If derivation of that file fails, then the file is *not* updated.
+  In principle, you'd expect the top-level `valueSetDate` field to be updated (to the date of today).
+  You could also inspect that file's metadata to be sure.
+4. Inspect the console.
+  The derivation prints issues (error, warnings, infos) there (if there are any).
+  If there are any manufacturers that are associated with multiple vaccines, a piece of JSON detailing those associations is printed to the console.
+5. Compare the updated file against the last version committed to the Git versioning system.
+  If there are more differences between these versions than expected, you need to have to look at the [`annex-A/table.tsv` file](./table.tsv) to see whether (and if so, how) that needs to be tweaked.
+  Issues printed to the console by running the script might help as well.
+
+The derivation relies on the Deno runtime for JavaScript/TypeScript.
+It can be found and downloaded at https://deno.land/.
+

annex-A/as-json.ts

@@ -0,0 +1,122 @@
+import {fileAsLines, readAsJson} from "./files.ts"
+import {groupBy} from "./functional.ts"
+
+
+const tableLines = fileAsLines("table.tsv")
+/*
+ * The contents of the 'table.tsv' file are copied from the table in Annex A of the guidelines document,
+ * after which some spurious newlines are cleaned up.
+ */
+
+
+const cleanUp = (str: string) => {
+    const match = str.match(/^"\s*(.+?)\s*"$/)
+    return match
+        ? match[1]
+        : str
+}
+
+
+type Mapping = {
+    "vaccine-code": string
+    "vaccine-manufacturer": string
+    "sct-codes": string[]
+    "note": string
+}
+
+const lineAsEncoding = (line: string): [string, Mapping] => {
+    const parts = line.split(/\t/).map(cleanUp)
+    return [
+        parts[0],
+        {
+            "vaccine-code": parts[1],
+            "vaccine-manufacturer": parts[2],
+            "sct-codes": /* parts[3] === undefined ? [] : */parts[3].split(/\s+or\s+/),
+            "note": parts[4]
+        }
+    ]
+}
+
+const keyedEncodings: [string, Mapping][] = tableLines
+    .slice(1)
+    .map(lineAsEncoding)
+
+
+// check against other value sets:
+const medicinalProduct = readAsJson("../vaccine-medicinal-product.json")
+const manufacturers = readAsJson("../vaccine-mah-manf.json")
+const prophylaxis = readAsJson("../vaccine-prophylaxis.json")
+
+const deprecationPostfix = " (deprecated)"
+keyedEncodings.forEach(([key, mapping], i) => {
+
+    const rowInfix = `@ mapping row ${i+2} for "${key}"`
+    const warnHeader = `[WARN] ${rowInfix}:`
+
+    const vaccineCode = mapping["vaccine-code"]
+    const withoutDeprecation = vaccineCode.endsWith(deprecationPostfix) ? vaccineCode.substring(0, vaccineCode.length - deprecationPostfix.length) : vaccineCode
+    if (vaccineCode.endsWith(deprecationPostfix)) {
+        console.info(`[INFO] ${rowInfix}: vaccine with code "${withoutDeprecation}" occurs even though the vaccine's been deprecated`)
+    }
+
+    if (!(withoutDeprecation in medicinalProduct.valueSetValues)) {
+        console.warn(`${warnHeader} vaccine code "${mapping["vaccine-code"]}" doesn't exist in value set "vaccines-covid-19-names"`)
+    }
+
+    if (!(mapping["vaccine-manufacturer"] in manufacturers.valueSetValues)) {
+        console.warn(`${warnHeader} vaccine manufacturer with ID "${mapping["vaccine-manufacturer"]}" doesn't exist in value set "vaccines-covid-19-auth-holders"`)
+    }
+
+    if (mapping["sct-codes"].length === 0) {
+        console.warn(`${warnHeader} no mapping to prophylaxes`)
+    }
+    if (new Set(...mapping["sct-codes"]).size < mapping["sct-codes"].length) {
+        console.warn(`${warnHeader} mappings to duplicate prophylaxes occur`)
+    }
+    mapping["sct-codes"].forEach((sctCode) => {
+        if (sctCode in prophylaxis.valueSetValues) {
+            const sct = prophylaxis.valueSetValues[sctCode]
+            if (!sct["active"]) {
+                console.warn(`${warnHeader} prophylaxis code "${sctCode}" is used despite it being deprecated`)
+            }
+        } else {
+            console.warn(`${warnHeader} prophylaxis code "${sctCode}" doesn't exist in value set "sct-vaccines-covid-19"`)
+        }
+    })
+
+})
+
+
+const manufacturersWithMultipleVaccines =
+    groupBy(keyedEncodings, ([_, mapping]) => mapping["vaccine-manufacturer"])
+        .filter(([_, encodings]) => encodings.length > 1)
+        .map(([key, encodings]) => [key, encodings.map(([_, mapping]) => mapping["vaccine-code"])])
+
+if (manufacturersWithMultipleVaccines.length > 0) {
+    console.log()
+    console.log(`[INFO] manufacturers with more than 1 vaccine:`)
+    console.dir(manufacturersWithMultipleVaccines)
+}
+
+
+// check for duplicate keys:
+const encodingsPerKey = groupBy(keyedEncodings, ([key]) => key)
+const duplicateKeyEntries = encodingsPerKey.filter(([_, group]) => group.length > 1)
+if (duplicateKeyEntries.length > 0) {
+    console.error(`[ERROR] key occurs more than once: ${duplicateKeyEntries.map(([key]) => `"${key}"`).join(", ")}`)
+    console.error(`    - (exiting before saving)`)
+    Deno.exit(1)
+}
+
+
+const valueSet = {
+    "valueSetId": "vaccines-covid-19-encoding-instructions",
+    "valueSetDate": new Date().toISOString().slice(0, 10),
+    "valueSetValues": Object.fromEntries(keyedEncodings)
+}
+
+Deno.writeTextFileSync(
+    "../vaccine-encoding-instructions.json",
+    JSON.stringify(valueSet, null, 2)
+)
+

annex-A/build.sh

@@ -0,0 +1 @@
+deno run --allow-read --allow-write as-json.ts

annex-A/files.ts

@@ -0,0 +1,16 @@
+export const fileAsString = (path: string): string =>
+    new TextDecoder().decode(Deno.readFileSync(path))
+
+
+export const fileAsLines = (path: string): string[] => {
+    const lines = fileAsString(path)
+        .split(/\n/)    // split on newlines
+    return lines[lines.length - 1] === ""
+        ? lines.slice(0, lines.length - 1)
+        : lines
+}
+
+
+export const readAsJson = (path: string): any =>
+    JSON.parse(fileAsString(path))
+

annex-A/functional.ts

@@ -0,0 +1,13 @@
+export const groupBy = <T>(ts: T[], keyFunc: (t: T) => string): [key: string, values: T[]][] => {
+    const map: { [key: string]: T[] } = {}
+    ts.forEach((t) => {
+        const key = keyFunc(t)
+        let group = map[key]
+        if (group === undefined) {
+            group = map[key] = []
+        }
+        group.push(t)
+    })
+    return Object.entries(map)
+}
+

annex-A/table.tsv

@@ -0,0 +1,39 @@
+Vaccine name	Vaccine code	Vaccine Manufacturer or MAH code	Vaccine type	Note
+Comirnaty	EU/1/20/1528	ORG-100030215	1119349007 or 28531000087107	The same code should be used also for adapted Comirnaty vaccines (such as bivalent Original/Omicron BA.1 and Original/Omicron BA.4-5) unless indicated otherwise.
+Spikevax	EU/1/20/1507	ORG-100031184	1119349007 or 28531000087107	"Previously COVID-19 Vaccine Moderna. The same code should be used also for adapted Spikevax vaccines (such as bivalent Original/Omicron BA.1) unless indicated otherwise."
+"Vaxzevria"	EU/1/21/1529	"ORG-100001699"	28531000087107 or 29061000087103	Use also for AstraZeneca covid-19 vaccines, except R-COVI, Covishield and Covid-19 (recombinant) by Fiocruz.
+Jcovden	EU/1/20/1525	ORG-100001417	28531000087107 or 29061000087103	Previously known as COVID-19 Vaccine Janssen.
+CVnCoV	CVnCoV	ORG-100006270	1119349007 or 28531000087107
+NVX-CoV2373	"NVX-CoV2373 (deprecated)"	ORG-100032020	28531000087107 or 1162643001	Do not use this code for new certificates, see entry for Nuvaxovid.
+Sputnik V	Sputnik-V	Gamaleya-Research-Institute	28531000087107 or 29061000087103
+Convidecia	Convidecia	ORG-100013793	28531000087107 or 29061000087103
+EpiVacCorona	EpiVacCorona	Vector-Institute	28531000087107 or 1162643001
+BBIBP-CorV	BBIBP-CorV	ORG-100020693	28531000087107 or 1157024006
+Inactivated SARS-CoV-2 (Vero Cell)	Inactivated-SARS-CoV-2-Vero-Cell	ORG-100010771	28531000087107 or 1157024006	Do not use this code for new certificates, see entry for WIBP-CorV.
+CoronaVac	CoronaVac	Sinovac-Biotech	28531000087107 or 1157024006	Do not use for Vacina adsorvida covid-19 (inativada) by Institito Butantan.
+Covaxin	Covaxin	Bharat-Biotech	28531000087107 or 1157024006	Also known as BBV152 A, B, C.
+Covishield	Covishield	ORG-100001981	28531000087107 or 29061000087103	Also known as ChAdOx1_nCoV-19.
+Covid-19 (recombinant)	Covid-19-recombinant	Fiocruz	28531000087107 or 29061000087103
+R-COVI	R-COVI	ORG-100007893	28531000087107 or 29061000087103
+CoviVac	CoviVac	Chumakov-Federal-Scientific-Center	28531000087107 or 1157024006
+Sputnik Light	Sputnik-Light	Gamaleya-Research-Institute	28531000087107 or 29061000087103
+Hayat-Vax	Hayat-Vax	ORG-100023050	28531000087107 or 1157024006
+Abdala	Abdala	CIGB	28531000087107	Also known as CIGB-66.
+WIBP-CorV	WIBP-CorV	Sinopharm-WIBP	28531000087107 or 1157024006	Previously known as Inactivated SARS-CoV-2 (Vero Cell).
+MVC COVID-19 vaccine	MVC-COV1901	ORG-100033914	28531000087107 or 1162643001	Also known as Medigen COVID-19 vaccine.
+Nuvaxovid	EU/1/21/1618	ORG-100032020	28531000087107 or 1162643001	Previously known as NVX-CoV2373.
+Covovax	Covovax	ORG-100001981	28531000087107 or 1162643001	Do not confuse with Nuvaxovid.
+Vidprevtyn	"Vidprevtyn (deprecated)"	ORG-100000788	28531000087107	Do not use this code for new certificates, see entry for VidPrevtyn Beta.
+VLA2001	"VLA2001 (deprecated)"	ORG-100036422	28531000087107	Do not use this code for new certificates, see entry for COVID-19 Vaccine (inactivated, adjuvanted) Valneva.
+Sputnik M	Sputnik-M	Gamaleya-Research-Institute	28531000087107 or 29061000087103	Do not confuse with Sputnik V or Sputnik Light.
+EpiVacCorona-N	EpiVacCorona-N	Vector-Institute	28531000087107 or 1162643001	Do not confuse with EpiVacCorona. Also know as Aurora-CoV.
+Vacina adsorvida covid-19 (inativada)	Covid-19-adsorvida-inativada	Instituto-Butantan	28531000087107 or 1157024006
+NVSI-06-08	NVSI-06-08	NVSI	28531000087107	Also known as Recombinant SARS-CoV-2 vaccine (CHO Cell).
+YS-SC2-010	YS-SC2-010	Yisheng-Biopharma	28531000087107	Also known as PIKA-Adjuvanted Recombinant SARS-CoV-2 Spike (S) Protein Subunit vaccine.
+SCTV01C	SCTV01C	ORG-100026614	28531000087107	Also known as Bivalent Recombinant Trimeric S Protein vaccine.
+Covifenz	Covifenz	ORG-100008549	28531000087107	Also known as CoVLP and Medicago plant-based VLP.
+AZD2816	AZD2816	"ORG-100001699"	28531000087107 or 29061000087103
+Soberana 02	Soberana-02	Finlay-Institute	28531000087107	Also known as FINLAY-FR-2.
+Soberana Plus	Soberana-Plus	Finlay-Institute	28531000087107	Also known as FINLAY-FR-1A.
+COVID-19 Vaccine Valneva	"EU/1/21/1624"	ORG-100036422	28531000087107	Previously known as VLA2001.
+VidPrevtyn Beta	EU/1/21/1580	ORG-100000788	28531000087107	Previously known as Vidprevtyn.

source-update/README.md

@@ -1,4 +1,4 @@
-# EU eHealthNetwork Digitial COVID Certificate RAT list update script
+# EU eHealthNetwork Digital COVID Certificate RAT list update script