Form Runner admin console with analytics

[ebruchez]

The purpose is to provide access to useful runtime information, including:

• number of compiled form definitions in memory
• form sessions
  • number of currently active
  • form sessions opened, with timestamp
  • with errors
  • submitted
  • abandoned
  • time spent
  • activity pattern (number of Ajax updates, for example)
  • time active/serialized
  • time to destruction (with session expiration)
  • whether currently active
• cache sizes
• available memory
• estimates of form memory usage
• timings
  • form compilation
  • form loads
  • form sessions initialization
  • calls to services
• etc.

Consider:

• Twitter Ostrich
• Metrics (seems more actively developed)

See also #1718.

[ebruchez]

Where to gather stats:

• caches
  • Caches.scala
  • XFormsStaticStateCache.scala
  • XPathCache.scala
  • XFormsDocumentCache.java

[ebruchez]

For now we don't need to schedule this, but:

• Work on Review XForms cache architecture #1718 might require some of this.
• There might be some work on this with a customer as well.

[ebruchez]

We have been doing some experimental work on this:

• logging data in JSON format
• started writing an analyzer to extract useful information

List of useful metrics [REVIEW]:

• connected users
  • non-expired servlet session
  • with activity in last n minutes
• active users
  • users with at least one active form
• forms that have had at least one interaction
• forms that have been sent
• forms that have been abandoned (not saved or sent)
• active forms over time, for some definition of active
  • haven't been saved but have been accessed during the last n minutes
• forms that have been loaded but that have never had an interaction
• number of concurrent requests
• number of pending requests (waiting in line to be served on one of the locks)
• processing times and latency
  • loading time (form + updates)
  • wait time (form + updates)
  • time to call services
  • time to call persistence layer
  • time to save/send
• activity over time, and the number of directions per minute
• email/PDF activity

[ebruchez]

As a tiny step, here is some information which would be useful:

• Orbeon Forms version (FR Home: in admin mode, show Orbeon Forms version number #2415)
  • shown at bottom of Home page when user has admin permissions
• Java properties including memory settings

As a start, we could have a tab/page in /fr/ to show this information when the user is logged.

[ebruchez]

+1 today to see what outgoing HTTP connections to services have happened and/or are pending. In particular, we have a case where a connection was stuck, and it would be good to know that this is happening (connection started but not yet completed, for example).

[ebruchez]

See also #2169.

[ebruchez]

• We could create a new /fr/console page for this. It could be split in parts/tabs as needed. /fr would remain as is for now.
• As a first step it could show memory settings and a few other important information.
• That page must only be shown to "admin" users.

[ebruchez]

See also #3534.

[ebruchez]

+1 from evaluator

[ebruchez]

I renamed to "admin console". In fact this is more about visibility. For development, we have #2169.

[ebruchez]

+1 from customer for:

• count/grouping of published forms by app name (improvement on Form Metadata API?)
• number of forms submitted by the user

[ebruchez]

We need to give this a new round of thinking as it would be a great enhancement to the product. This could be placed either as a tab in the /fr/admin page, or as a separate page, like /fr/console. This would initially require the same access rights as the Admin page.

Metrics fall under different categories:

• system, like memory, etc.
• other low-level, like connections, etc.
• low-level form status, like caches
• high-level form status, like forms filled, etc.

As a first step, we need to see if something like Metrics would really help (I suspect it would).

Then we can look into implementing support for the first metrics, and go on from there.

[avernet]

+1 from customer