WIP: api18970: modifications per discussion 2019-11-19

See mbolivar comment at
zephyrproject-rtos#18970 (comment)
and subsequent discussion.

Add Background section to reference and summarize key concepts from
existing documentation, particularly for thread state and priority and
execution context.

Replaced "blocking" with "yielding" for precision and consistency with
Zephyr thread terminology.

Add Commentary subsections that give a bit of explanation for why
something is important or how concepts relate to each other.

Signed-off-by: Peter Bigot <[email protected]>

doc/api18970.md

@@ -4,13 +4,91 @@
 
 [#18970]: https://github.com/zephyrproject-rtos/zephyr/issues/18970
 
+## Background
+
+See also the Zephyr [glossary][].
+
+[glossary]: https://docs.zephyrproject.org/latest/glossary.html
+
+### Thread Terminology
+
+Zephyr defines [six thread
+states](https://docs.zephyrproject.org/latest/reference/kernel/threads/index.html#thread-states)
+of which four are active:
+
+* **Ready** when there is nothing that prevents the thread from becoming
+  active as the current thread, but it is not the current thread.
+* **Running** when the thread is active as the current thread (on a
+  processor).
+* **Waiting** when the thread is on a queue waiting for an event to occur
+  that will transition it to *Ready*.
+* **Suspended** when the thread is inactive and must be transitioned to
+  *Ready* through explicitly (via `k_thread_resume()`.
+
+We shall say that a thread **yields** when it invokes a function that
+causes it to become **unready** but not terminated: that is, it
+transitions to *Waiting* or *Suspended*.
+
+**NOTE** The term "sleeping" when applied to Zephyr threads has
+historically mean a thread that is *Waiting* for a timeout event.
+
+Zephyr defines [two classes of
+threads](https://docs.zephyrproject.org/latest/reference/kernel/threads/index.html#thread-priorities):
+
+* **cooperative** threads remain the current thread until they are made
+  unready.
+* **preemptible** threads remain the current thread until either (a) a
+  cooperative thread becomes *Ready*, or (b) a higher-priority
+  preemptible thread becomes *Ready*.
+
+For API behavior we are primarily interested in whether invoking a
+function can cause a cooperative thread to yield.
+
+### Context Terminology
+
+**TODO** fill this out
+
+This section is intended to describe the privilege and processor context
+variations in which a thread can run.  It should provide or reference
+definitions of terms like this:
+
+* Kernel [initialization
+  level](https://docs.zephyrproject.org/latest/reference/drivers/index.html#initialization-levels)
+* User-space versus kernel (system call)
+* Normal ("thread"?) versus interrupt (invoked from an Interrupt Service
+  Routine)
+
+For API behavior we are interested in whether a particular function
+**may**, **must**, or **must not** be invoked from a specific context.
+
 ## Definitions
 
+### yield (thread)
+
+A thread yields when it invokes an operation that transitions it to
+*Suspended* or *Waiting*.
+
+### pre-empted (thread)
+
+A thread is pre-empted when it transitions from *Running* to *Ready*
+without having yielded.  (Only preemptible threads can be pre-empted.)
+
+### yielding (function)
+
+A function is yielding when it invokes (directly or indirectly) an
+operation that causes the current thread to yield.
+
+A function is non-yielding when it is not yielding.
+
 ### thread-safe
 
 A function is thread-safe if its behavior is correct when invoked from
 multiple threads at the same time.
 
+#### Commentary
+
+Most public API will satisfy this condition; some private API may not.
+
 ### reentrant
 
 A function is reentrant if its behavior is correct when it is invoked by
@@ -21,13 +99,77 @@ A function is reentrant if its behavior is correct when it is invoked by
 A function is interrupt-safe if its behavior is not affected by
 concurrent access from interrupts.
 
-### block
+#### Commentary
+
+Most public API will satisfy this condition; some private API may not.
+
+We need to be able to say succinctly "Unless otherwise specified all API
+functions are interrupt-safe" and expect people to know what that means.
+A specific example would be the GPIO API.  Because GPIO write functions
+may be invoked from ISRs read-modify-write code like:
+
+```
+u32_t out = gpio->OUT;
+gpio->OUT ^= (out & ~mask) | (value & mask);
+```
+
+*must* be wrapped in a spin-lock to be interrupt-safe.  Many current
+implementations do not satisfy this requirement.
 
-A call blocks if it can suspend the invoking thread while it waits for
-something to happen.
+On the other hand internal functions may be written to assume they are
+called with interrupts disabled, or a specific lock held.
+
+**TODO** standard marking?
 
 ### atomic
 
 An operation is atomic if the steps it makes internally cannot be
 affected by nor visible to interleaving executions, such as from
 interrupts or thread pre-emption.
+
+An operation that is atomic is by definition interrupt-safe.
+
+An operation that is atomic is by definition thread-safe.
+
+### asynchronous
+
+A function is asynchronous if it may return before the operation it
+initiates has completed.  An asynchronous function must provide a
+mechanism by which completion is reported, e.g. a callback, event, or
+signal.
+
+#### Commentary
+
+Note that asynchronous is orthogonal to non-yielding.  Some API may
+provide completion information through a callback, but may yield while
+waiting.
+
+### queued
+
+A function is queued if it is asynchronous and allows multiple
+operations to be outstanding at any time.
+
+#### Commentary
+
+This concept is necessary because of operations like
+`spi_transceive_async()` which returns its result through a signal, but
+will yield if the device is already processing an asynchronous
+operation.
+
+Queued operations are rare(? do not exist?) in Zephyr, and require that
+the API use a chainable persisted state object to hold the operation
+parameters in a persisted state object that can be added to an internal
+queue for processing when the required resource is available.
+
+## Other Rules
+
+* A function that cannot fulfill its contract when invoked from
+  interrupt context must indicate this failure, generally through
+  returning `-EWOULDBLOCK`.
+
+## To Do
+
+- [ ] Consider a standard marking for private functions that must be
+      invoked with an held or interrupts disabled, such as a suffix
+      `_locked`.
+- [ ] Define the terminology related to execution context