Merge pull request #2076 from postscript-dev/update_docs

Update markdown documentation

README.md

@@ -910,6 +910,7 @@ The Variable EXIV2\_PORT or EXIV2\_HTTP can be set to None to skip http tests.
 
 [TOC](#TOC)
 <div id="4-1">
+
 ### 4.1 Running tests on Unix-like systems
 
 You can run tests directly from the build:
@@ -945,6 +946,7 @@ $
 
 [TOC](#TOC)
 <div id="4-2">
+
 ### 4.2 Running tests on Visual Studio builds from cmd.exe
 
 **Caution:** _The python3 interpreter must be on the PATH, build for DOS, and called python3.exe._  I copied the python.exe program:
@@ -995,6 +997,7 @@ set EXIV2_PORT=
 
 [TOC](#TOC)
 <div id="4-3">
+
 ### 4.3 Unit Tests
 
 The code for the unit tests is in `<exiv2dir>/unitTests`.  To include unit tests in the build, use the *cmake* option `-DEXIV2_BUILD_UNIT_TESTS=On`.
@@ -1014,6 +1017,7 @@ $ popd
 
 [TOC](#TOC)
 <div id="4-4">
+
 ### 4.4 Bugfix Tests
 
 You can run the bugfix tests from the build directory:

tests/README-TESTS.md

@@ -3,6 +3,7 @@
 - [README-TESTS](#README-TESTS)
   - [Running the test suite](#running-the-test-suite)
   - [Writing new tests](#writing-new-tests)
+  - [Test data](#test-data)
   - [Test suite](#test-suite)
     - [Configuration](#configuration)
       - [INI style](#ini-style)
@@ -25,6 +26,7 @@
 
 
 <div id="README-TESTS"/>
+
 # README-TESTS
 
 This test suite is intended for system tests, i.e. for running a binary with
@@ -34,6 +36,7 @@ of new features where unit testing is not feasible, e.g. to test new command
 line parameters.
 
 <div id="running-the-test-suite"/>
+
 ## Running the test suite
 
 The test suite is written for Python 3 and is not compatible with Python 2, thus
@@ -65,6 +68,7 @@ standard output.
 [TOC](#TOC)
 
 <div id="writing-new-tests"/>
+
 ## Writing new tests
 
 The test suite is intended to run a binary and compare its standard output,
@@ -111,7 +115,22 @@ When creating new tests, follow roughly these steps:
 
 [TOC](#TOC)
 
+<div id="test-data"/>
+
+## Test data
+All test files (e.g. images, `.exv`, etc) are stored in the `$data_path` 
+directory, with any temporary copies in the `$tmp_path` directory (see 
+[Creating file copies](#creating-file-copies)).
+
+When writing tests, try to reuse existing data files rather than add new ones. 
+If new files are required, prefer adding only the metadata (`.exv`) instead of 
+the whole image. These steps help to reduce the size of the test data 
+directory.
+
+[TOC](#TOC)
+
 <div id="test-suite"/>
+
 ## Test suite
 
 The test suite itself uses the builtin `unittest` module of Python to discover
@@ -131,6 +150,7 @@ group test cases.
 ### Configuration
 
 <div id="ini-style"/>
+
 #### INI style
 
 The test suite is configured via `INI` style files using Python's builtin
@@ -181,6 +201,7 @@ some_var:some value with whitespaces before and after
 [TOC](#TOC)
 
 <div id="parameters"/>
+
 #### Parameters
 
 The test suite's configuration file should have the following form:
@@ -276,6 +297,7 @@ variable names in Python.
 [TOC](#TOC)
 
 <div id="test-cases"/>
+
 ### Test cases
 
 The test cases are defined in Python source files utilizing the unittest module,
@@ -363,6 +385,7 @@ writing 0 ten times. The same is of course possible for strings.
 [TOC](#TOC)
 
 <div id="multiline-strings"/>
+
 ### Multiline strings
 
 It is generally recommended to use Python's multiline strings (strings starting
@@ -401,6 +424,7 @@ character. To achieve that put the `"""` on the following line.
 [TOC](#TOC)
 
 <div id="paths"/>
+
 ### Paths
 
 Some test cases require the specification of paths (e.g. to the location of test
@@ -430,13 +454,15 @@ class AnInformativeName(metaclass=CaseMeta):
 [TOC](#TOC)
 
 <div id="advanced-test-cases"/>
+
 ## Advanced test cases
 
 This section describes more advanced features that are probably not necessary
 the "standard" usage of the test suite.
 
 
 <div id="providing-standard-input-to-commands"/>
+
 ### Providing standard input to commands
 
 The test suite supports providing a standard input to commands in a similar
@@ -483,6 +509,7 @@ test suite will implicitly assume `None` for every command.
 [TOC](#TOC)
 
 <div id="using-a-different-output-encoding"/>
+
 ### Using a different output encoding
 
 The test suite will try to interpret the program's output as utf-8 encoded
@@ -526,6 +553,7 @@ encodings can be found
 [TOC](#TOC)
 
 <div id="working-with-binary-output"/>
+
 ### Working with binary output
 
 Some programs output binary data directly to stdout or stderr. Such programs can
@@ -595,28 +623,30 @@ and your test case will get only the specified environment variables.
 [TOC](#TOC)
 
 <div id="creating-file-copies"/>
+
 ### Creating file copies
 
 For tests that modify their input file it is useful to run these with a
-disposable copy of the input file and not with the original. For this purpose
+disposable copy of the input file and not with the original. For this purpose,
 the test suite features a decorator which creates a copy of the supplied files
-and deletes the copies after the test ran.
+and preserves the copies after the test finishes. All temporary copies are 
+stored in the `$tmp_path` directory.
 
 Example:
 
 ```python
 # -*- coding: utf-8 -*-
 
-import system_tests
+import system_tests, CopyTmpFiles
 
 
-@system_tests.CopyFiles("$filename", "$some_path/another_file.txt")
+@CopyTmpFiles("$data_path/invalid_input_file.txt")
 class AnInformativeName(metaclass=system_tests.CaseMeta):
 
-    filename = "invalid_input_file"
+    filename = path("$tmp_path/invalid_input_file.txt")
     commands = [
-	    "$binary -c $import_file -i $filename"
-	]
+        "$binary --option $filename"
+    ]
     retval = ["$abort_exit_value"]
     stdout = ["Reading $filename"]
     stderr = [
@@ -626,16 +656,16 @@ error in $filename
     ]
 ```
 
-In this example, the test suite would automatically create a copy of the files
-`invalid_input_file` and `$some_path/another_file.txt` (`some_path` would be of
-course expanded too) named `invalid_input_file_copy` and
-`$some_path/another_file_copy.txt`. After the test ran, the copies are
-deleted. Please note that variable expansion in the filenames is possible.
+In this example, the test suite would automatically create a copy of the file 
+`$data_path/invalid_input_file.txt` called `$tmp_path/invalid_input_file.txt`. 
+After the test runs, the temporary copy is preserved. Please note that variable 
+expansion in the filenames is possible.
 
 
 [TOC](#TOC)
 
 <div id="customizing-the-output-check"/>
+
 ### Customizing the output check
 
 Some tests do not require a "brute-force" comparison of the whole output of a
@@ -694,6 +724,7 @@ class AnInformativeName(metaclass=system_tests.CaseMeta):
 [TOC](#TOC)
 
 <div id="running-all-commands-under-valgrind"/>
+
 ### Running all commands under valgrind
 
 The test suite can run all commands under a memory checker like
@@ -736,6 +767,7 @@ into account:
 [TOC](#TOC)
 
 <div id="manually-expanding-variables"/>
+
 ### Manually expanding variables in strings
 
 In case completely custom checks have to be run but one still wants to access
@@ -791,6 +823,7 @@ will result in `another_string` being "foo" and not "bar".
 [TOC](#TOC)
 
 <div id="hooks"/>
+
 ### Hooks
 
 The `Case` class provides two hooks that are run after each command and after
@@ -848,6 +881,7 @@ class AnInformativeName(metaclass=system_tests.CaseMeta):
 [TOC](#TOC)
 
 <div id="bash-test-cases"/>
+
 ## Bash test cases
 
 - Previously, Exiv2 had some bash test scripts, which were saved as the file `EXIV2_DIR/test/*.sh`. We're going to rewrite them as Python test scripts and save them to the directory `EXIV2_DIR/tests/bash_tests`.