Clarify description of structured vs unstructured meta-information lines

VCFv4.1.tex

@@ -136,7 +136,7 @@ \subsubsection{Pedigree field format}
 \end{verbatim}
 or a link to a database:
 \begin{verbatim}
-##pedigreeDB=<url>
+##pedigreeDB=URL
 \end{verbatim}
 \subsection{Header line syntax}
 The header line names the 8 fixed, mandatory columns. These columns are as follows:
@@ -893,7 +893,7 @@ \subsubsection{Clonal derivation relationships}
 Alternately, if data on the genomes is compiled in a database, a simple pointer can be provided:
 
 \begin{verbatim}
-##pedigreeDB=<url>
+##pedigreeDB=URL
 \end{verbatim}
 
 The most general form of a pedigree line is:

VCFv4.2.tex

@@ -153,7 +153,7 @@ \subsubsection{Pedigree field format}
 \end{verbatim}
 or a link to a database:
 \begin{verbatim}
-##pedigreeDB=<url>
+##pedigreeDB=URL
 \end{verbatim}
 \subsection{Header line syntax}
 The header line names the 8 fixed, mandatory columns. These columns are as follows:
@@ -910,7 +910,7 @@ \subsubsection{Clonal derivation relationships}
 Alternately, if data on the genomes is compiled in a database, a simple pointer can be provided:
 
 \begin{verbatim}
-##pedigreeDB=<url>
+##pedigreeDB=URL
 \end{verbatim}
 
 The most general form of a pedigree line is:

VCFv4.3.tex

@@ -100,24 +100,38 @@ \subsection{Data types}
 For the Integer type, the values from $-2^{31}$ to $-2^{31}+7$ cannot be stored in the binary version and therefore are disallowed in both VCF and BCF, see \ref{BcfTypeEncoding}.
 
 \subsection{Meta-information lines}
-File meta-information is included after the \#\# string and must be key=value pairs.
-Meta-information lines are optional, but if they are present then they must be completely well-formed.
-Note that BCF, the binary counterpart of VCF, requires that all entries are present.
-It is recommended to include meta-information lines describing the entries used in the body of the VCF file.
+File meta-information lines start with ``\verb|##|'' and must appear first in the VCF file, before the header line (section~\ref{header-line}) and data record lines (section~\ref{data-lines}).
+They may be either \emph{unstructured} or \emph{structured}.
+
+An \emph{unstructured} meta-information line consists of a~\emph{key} (denoting the type of meta-information recorded) and a~\emph{value} (which may not be empty and must not start with a `\verb|<|' character), separated by an `\verb|=|' character:
+\begin{quote}
+\verb|##|\emph{key}\verb|=|\emph{value}
+\end{quote}
+Several unstructured meta-information lines are defined in this specification, notably \verb|##fileformat|.
+Others not defined by this specification, e.g.\ \verb|##fileDate| and \verb|##source|, are commonly found in VCF files.
+These typically have meanings that are obvious, or they are immaterial for processing the file, or both.
 
-All structured lines that have their value enclosed within ``$<>$'' require an ID which must be unique within their type.
-For all of the structured lines (\#\#INFO, \#\#FORMAT, \#\#FILTER, etc.), extra fields can be included after the default fields.
+A \emph{structured} meta-information line is similar, but the value is itself a comma-separated list of key=value pairs, enclosed within `\verb|<|' and `\verb|>|' characters:
+\begin{quote}
+\verb|##|\emph{key}\verb|=<|\emph{key}\verb|=|\emph{value}\verb|,|\emph{key}\verb|=|\emph{value}\verb|,|\emph{key}\verb|=|\emph{value}\verb|,|\ldots\verb|>|
+\end{quote}
+All structured lines require an ID which must be unique within their type, i.e., within all the meta-information lines with the same ``\verb|##|\emph{key}\verb|=|'' prefix.
+For all of the structured lines (\verb|##INFO|, \verb|##FORMAT|, \verb|##FILTER|, etc.) described in this specification, extra fields can be included after the default fields.
 For example:
 \begin{verbatim}
-##INFO=<ID=ID,Number=number,Type=type,Description="description",Source="description",Version="128">
+##INFO=<ID=ID,Number=number,Type=type,Description="description",Source="source",Version="128">
 \end{verbatim}
 In the above example, the extra fields of ``Source'' and ``Version'' are provided.
-Optional fields must be stored as strings even for numeric values.
+The values of optional fields must be written as quoted strings, even for numeric values.
+Other structured lines not defined by this specification may also be used; the only default field for such lines is the required \verb|ID| field.
 
 It is recommended in VCF and required in BCF that the header includes tags describing the reference and contigs backing the data contained in the file.
 These tags are based on the SQ field from the SAM spec; all tags are optional (see the VCF example above).
 
-Meta-information lines can be in any order with the exception of `fileformat` which must come first.
+Meta-information lines are optional, but if they are present then they must be completely well-formed.
+Other than \verb|##fileformat|, they may appear in any order.
+Note that BCF, the binary counterpart of VCF, requires that all entries are present.
+It is recommended to include meta-information lines describing the entries used in the body of the VCF file.
 
 
 \subsubsection{File format}
@@ -266,6 +280,7 @@ \subsubsection{Pedigree field format}
 
 
 \subsection{Header line syntax}
+\label{header-line}
 The header line names the 8 fixed, mandatory columns. These columns are as follows:
 \begin{center}
        \#CHROM
@@ -1306,7 +1321,7 @@ \subsubsection{Clonal derivation relationships}
 Alternately, if data on the genomes is compiled in a database, a simple pointer can be provided:
 
 \begin{verbatim}
-##pedigreeDB=<url>
+##pedigreeDB=URL
 \end{verbatim}
 
 \begin{samepage}

test/vcf/4.1/failed/failed_meta_pedigreedb_002.vcf

@@ -1,5 +1,5 @@
 ##fileformat=VCFv4.1
 ##CauseOfFailure=Non-valid URL
-##pedigreeDB=<ftp://8080:8080/not-valid/host/to/pedigreeDB>
+##pedigreeDB=ftp://8080:8080/not-valid/host/to/pedigreeDB
 #CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO
 1	123	.	TC	T	.	.	.

test/vcf/4.1/passed/complexfile_passed_000.vcf

@@ -32,7 +32,7 @@
 ##assembly=ftp://user@host:8080/path/to/file.fastq
 ##PEDIGREE=<Name_0=Something>
 ##PEDIGREE=<Name_0=Something,Name_1=Something-else>
-##pedigreeDB=<ftp://user@host:8080/path/to/pedigreeDB?arg1=db1>
+##pedigreeDB=ftp://user@host:8080/path/to/pedigreeDB?arg1=db1
 ##contig=<ID=1>
 ##contig=<ID=contig_url,URL=ftp://user@host:8080/path/to/contig>
 ##contig=<ID=contig_accession,species="Homo sapiens",accession=GCA_000001405.1>

test/vcf/4.1/passed/passed_meta_pedigreedb.vcf

@@ -1,5 +1,5 @@
 ##fileformat=VCFv4.1
-##pedigreeDB=<ftp://www.ebi.ac.uk:8080/valid/host/to/file.db>
-##pedigreeDB=<http://123.0.1.2:8080/valid/host/to/file.db>
+##pedigreeDB=ftp://www.ebi.ac.uk:8080/valid/host/to/file.db
+##pedigreeDB=http://123.0.1.2:8080/valid/host/to/file.db
 #CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO
 1	123	.	TC	T	.	.	.

test/vcf/4.2/failed/failed_meta_pedigreedb_002.vcf

@@ -1,5 +1,5 @@
 ##fileformat=VCFv4.2
 ##CauseOfFailure=Non-valid URL
-##pedigreeDB=<ftp://8080:8080/not-valid/host/to/pedigreeDB>
+##pedigreeDB=ftp://8080:8080/not-valid/host/to/pedigreeDB
 #CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO
 1	123	.	TC	T	.	.	.

test/vcf/4.2/passed/complexfile_passed_000.vcf

@@ -32,7 +32,7 @@
 ##assembly=ftp://user@host:8080/path/to/file.fastq
 ##PEDIGREE=<Name_0=Something>
 ##PEDIGREE=<Name_0=Something,Name_1=Something-else>
-##pedigreeDB=<ftp://user@host:8080/path/to/pedigreeDB?arg1=db1>
+##pedigreeDB=ftp://user@host:8080/path/to/pedigreeDB?arg1=db1
 ##contig=<ID=1>
 ##contig=<ID=contig_url,URL=ftp://user@host:8080/path/to/contig>
 ##contig=<ID=contig_accession,species="Homo sapiens",accession=GCA_000001405.1>

test/vcf/4.2/passed/passed_meta_pedigreedb.vcf

@@ -1,5 +1,5 @@
 ##fileformat=VCFv4.2
-##pedigreeDB=<ftp://www.ebi.ac.uk:8080/valid/host/to/file.db>
-##pedigreeDB=<http://123.0.1.2:8080/valid/host/to/file.db>
+##pedigreeDB=ftp://www.ebi.ac.uk:8080/valid/host/to/file.db
+##pedigreeDB=http://123.0.1.2:8080/valid/host/to/file.db
 #CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO
 1	123	.	TC	T	.	.	.

test/vcf/4.3/failed/failed_meta_pedigreedb_002.vcf

@@ -1,5 +1,5 @@
 ##fileformat=VCFv4.3
 ##CauseOfFailure=Non-valid URL
-##pedigreeDB=<ftp://8080:8080/not-valid/host/to/pedigreeDB>
+##pedigreeDB=ftp://8080:8080/not-valid/host/to/pedigreeDB
 #CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO
 1	123	.	TC	T	.	.	.

test/vcf/4.3/passed/complexfile_passed_000.vcf

@@ -32,7 +32,7 @@
 ##assembly=ftp://user@host:8080/path/to/file.fastq
 ##PEDIGREE=<ID=Pedigree1,Original=Something>
 ##PEDIGREE=<ID=Pedigree2,Name_0=Something,Name_1=Something-else>
-##pedigreeDB=<ftp://user@host:8080/path/to/pedigreeDB?arg1=db1>
+##pedigreeDB=ftp://user@host:8080/path/to/pedigreeDB?arg1=db1
 ##contig=<ID=1>
 ##contig=<ID=contig_url,URL=ftp://user@host:8080/path/to/contig>
 ##contig=<ID=contig_accession,species="Homo sapiens",accession=GCA_000001405.1>

test/vcf/4.3/passed/passed_meta_pedigreedb.vcf

@@ -1,5 +1,5 @@
 ##fileformat=VCFv4.3
-##pedigreeDB=<ftp://www.ebi.ac.uk:8080/valid/host/to/file.db>
-##pedigreeDB=<http://123.0.1.2:8080/valid/host/to/file.db>
+##pedigreeDB=ftp://www.ebi.ac.uk:8080/valid/host/to/file.db
+##pedigreeDB=http://123.0.1.2:8080/valid/host/to/file.db
 #CHROM	POS	ID	REF	ALT	QUAL	FILTER	INFO
 1	123	.	TC	T	.	.	.