Derived Values

To follow along with this section, start with tag v2.0.5.

Transform functions receive messages and apply them to the data model. What if you need to have a value in the data model that is based on values which are modified by more than one transform function?

Derive functions allow you to compute new values from any other values in the data model. A derive function will be called when any of its inputs change. Derive functions can be arranged into an arbitrary dataflow.

In this step you will calculate some additional information based on the value of the local counter and any additional counter values. While doing this you will be introduced to the following pedestal-app concepts:

• Derive functions
• Dataflow
• Working with dataflow inputs

Remember to have the dev tools running before you start.

Deriving a total

One interesting value to have would be the total of all the counters. Add the function below to the tutorial-client.behavior namespace.

(defn total-count [_ nums] (apply + nums))

Except for the fact that it has two arguments, the first of which we are ignoring, this looks like a normal function to calculate the sum of a sequence of numbers.

A derive function is applied to some location in the data model. The value at that location is passed as the first argument to the function and the return value will become the new value at that location.

The total-count function always produces a new value so the old value is ignored.

The remaining arguments to a derive function depend on how that function is configured.

Derive functions are configured in the dataflow definition by adding a set of configuration vectors under the key :derive.

:derive #{[#{[:my-counter] [:other-counters :*]} [:total-count] total-count :vals]}

A set is used because order is not important. A function order will be determined at run time based on inputs and outputs.

Each derive configuration vector can have three or four elements.

[inputs output-path derive-fn input-spec] ;; input-spec is optional

In this example, a set is used to describe the inputs to this function. The function will receive the value at :my-counter and each of the :other-counter values. The output path is [:total-count], the function is total-count and the input specifier is :vals.

The input specifier describes how the arguments will be passed to the function. In this case, all of the values will be put into a single collection and passed to the function. Other options for the input specifier are:

:single-val
:map
:map-seq
:default

If you don't use an input specifier, it is the same as using :default. In this case the function will be passed the inputs map from which any information about the data model and what has changed can be collected.

When this derive function runs it will produce a new value in the data model under the key :total-count. You will need to update the default emitter to report change at this location.

[#{[:my-counter]
   [:other-counters :*]
   [:total-count]} (app/default-emitter [:main])]

If you now refresh the Data UI, you should see the :total-count and see it update as the other counters change.

Deriving a maximum count

To illustrate how to make use of the old data value, add a function to calculate the maximum counter value.

Create the derive function.

(defn maximum [old-value nums]
  (apply max (or old-value 0) nums))

Notice that this derive function makes use of the old-value.

As you did above, add this derive function to the dataflow definition

:derive #{[#{[:my-counter] [:other-counters :*]} [:total-count] total-count :vals]
          [#{[:my-counter] [:other-counters :*]} [:max-count] maximum :vals]}

and allow these changes to be reported.

[#{[:my-counter]
   [:other-counters :*]
   [:total-count]
   [:max-count]} (app/default-emitter [:main])]

The maximum value should now be displayed in the Data UI.

Deriving the average count

Derive functions can have inputs which are the results of other derive functions. To demonstrate this, create a derive function which will calculate the average of all the counters. This will use the :total-count value which has already been calculated.

(defn average-count [_ {:keys [total nums]}]
  (/ total (count nums)))

The average-count function receives a map as an argument which is destructured to get the total and nums values. To supply the arguments in this way, use a map instead of a set to configure the inputs.

:derive #{[#{[:my-counter] [:other-counters :*]} [:total-count] total-count :vals]
          [#{[:my-counter] [:other-counters :*]} [:max-count] maximum :vals]

          [{[:my-counter] :nums
            [:other-counters :*] :nums
            [:total-count] :total}
           [:average-count] average-count :map]}

Using a map to describe inputs allows you to give useful names to the keys in the argument map. The configuration above will cause a map to be passed to the :average-count function where the keys in the map are :nums and :total. Notice that two of the entries have the same key. This will cause all of the values of the other counters plus the value of :my-counter to be stored under the same key as a set.

Finally, add :average-count to the default-emitter.

[#{[:my-counter]
   [:other-counters :*]
   [:total-count]
   [:max-count]
   [:average-count]} (app/default-emitter [:main])]

Refreshing the page should now show the calculated average.

Simplify with a single list of counters

In many cases it would be much simpler to have all the counters in a single list. In others it is good to have them separated. We can use a derive function to create a single list without destroying the original values.

(defn merge-counters [_ {:keys [me others]}]
  (assoc others "Me" me))

The merge-counters function will take the local counter as me and others, which are a map, and assoc the local counter into the map under "Me".

Configure the new derive function.

[{[:my-counter] :me [:other-counters] :others} [:counters] merge-counters :map]

The input [:other-counters] refers to a map. In previous configurations, the individual counters were referenced with [:other-counters :*]. The latter version produces several individual numbers while the former produces a map.

Now that this single list of counters exists at [:counters], the other derive configurations can be simplified.

:derive #{[{[:my-counter] :me [:other-counters] :others} [:counters] merge-counters :map]
          [#{[:counters :*]} [:total-count] total-count :vals]
          [#{[:counters :*]} [:max-count] maximum :vals]
          [{[:counters :*] :nums [:total-count] :total} [:average-count] average-count :map]}

Dataflow

The derive functions above calculate three new values. This could all be done in transform functions. Why dataflow?

Dataflow helps you to reduce coupling in a program. Notice that each of the derive functions above don't know anything about where the input data comes from or where the output goes. This makes the code simpler, more reusable and less likely to change.

With dataflow programming, when you need to add new features, you tend to add new functions instead of changing existing ones. This makes code easier to maintain over time.

Because dataflow functions are small and loosely coupled to the things they depend on, they tend to be more reusable.

This is all very nice, but the best thing about dataflow is that it allows you to write all of your behavior code as pure functions. You can think of the dataflow engine as a giant Clojure reference type, like agents or refs. You supply pure state transition functions and it handles all of the complexities of managing state.

Next steps

The application's behavior is almost finished. In the next section you will see how to make the application a bit more interesting by showing some data about how the dataflow engine is performing.

The tag for this section is v2.0.6.

This section links to two pages. Debug Messages is an optional page and may be skipped. If you do skip this page, make sure to checkout the tag v2.0.7 before continuing on to Post Processing.

Home | Debug Messages | Post Processing