Collecting use cases

[dmlloyd]

As a prerequisite to deciding what to implement, and how, we need to understand who our users are (or might be) and what kinds of things they may want to be able to accomplish. Once we have done so, we can follow up by deciding which of these use cases we want to directly support, which use cases we want to indirectly allow, and which use cases we want to forbid.

The fastest and best way to accomplish this is to start by brainstorming the user roles concurrently to the use cases we wish to enable.

The way brainstorming works is that we collectively come up with as many ideas as possible without filtering them down. In this respect, no idea is "wrong"; merely suggesting a use case is not a guarantee that the use case must be implemented. That is decided later. If someone does not like a specific use case and thinks it should be worded differently or varied in some way then they can simply suggest another use case that is the way they want.

To start things off, I would begin with the following roles:

• Configuration file author - this is a user who is authoring an actual configuration file.
• Configuration consumer - this is a user who is trying to consume the configuration in some way.
• Configuration implementer - this is a user who is implementing some means for acquiring configuration.

And a couple of sample use cases, in the form of a simple user story:

• As a configuration file author, I want to have a means to know what configuration keys will be recognized so that they can avoid errors arising from key name mistakes.
• As a configuration implementer, I want to have the ability to easily and accurately reflect a flat key-value property format so that they can quickly support custom configuration formats.

We should collect these stories and categorize them by role, and then further categorize them by shared characteristics. The stories should cover topics such as tree and flat structure, mutability, verification, conversion, configuration formats, compatibility, etc.

Once we've collected a good number of these use cases, we can decide as a group which use cases should be supported by the implementation. The next step after that would be going back to the use cases that we do not want to support and deciding as a group which of these should be actively disallowed. Or we can approach it some other way, depending on the kinds of use cases we gather and how we all feel about them.

By starting with use cases, we will be able to make forward progress since we don't have to reach any consensus in order to add use cases, and we can remove them by simple majority after we are done - it could be as easy as a Google poll after some initial discussion period (this would IMO be up to Emily and Dmitry to decide). Once we have agreed on the use cases, then we can design the spec and API with a minimum of argument, knowing that it only does what was agreed upon, and the only real discussion that should be necessary at that point will relate specifically to code and documentation practices - in other words we should not deadlock at that point, and discussions should move things forward rather than round and round. And we'll have a cogent record of what led us to the final result.

The downside is that it may be a little tedious spelling it all out. But I think if we work together we should be able to get through the initial process fairly quickly.

[m-reza-rahman]

Slide 25 of my Jakarta EE 10 deck discusses the higher level use cases I would love to see addressed in the long and short term. Slide 26 contains some examples. Happy to try to translate these in whatever form is useful here.

Here is a first attempt:

• As a configuration consumer, I would like to externalize some aspects of my application configuration.
• As a configuration consumer, I expect support for some sensible built-in configuration sources such as property files, Java system properties and environment variables.
• As a configuration consumer, I expect properly files will reside in a default location that can be overridden.
• As a configuration consumer, I expect there will be common vendor provided plugins that support additional configuration sources such as Kubernetes secrets, secure cloud storage (e.g. the Azure Key Vault or Azure Configuration Service), NoSQL stores (e.g. Azure Redis). Ideally writing these plugins should not be too cumbersome.
• As a configuration consumer, I largely expect configuration to be stored in key-value format rather than XML, YAML, JSON, etc. Support for more complex storage formats is a fairly distant nice to have rather than a must have.
• As a configuration consumer, I mostly expect that configuration values should be treated as strings by default (or objects if truly necessary). I also expect some sensible set of built-in converters for Java primitives/wrappers and common date/time types. Furthermore, I expect to be able to provide my own custom converters. Plugin providers should probably also be able to provide their own converters.
• As a configuration consumer, I expect to be able to discern if a configuration value was left unspecified as opposed to being purposely specified as empty. An exception to this is when I have explicitly set a default value.
• As a configuration consumer, I expect that an update to a configuration value should be reflected in the application when possible (e.g. while doing a post-deployment programmatic lookup). This is a very common scenario on the cloud such as on Azure.
• As a configuration consumer, I expect to be able to inject configuration values using the @Inject annotation.
• As a configuration consumer, I expect to be able to reference configuration values in EL. This is a longer term expectation.
• As a configuration consumer, I expect to be able to reference configuration values in Jakarta EE deployment descriptors such as web.xml, persistence.xml, etc. This is a longer term expectation.
• As a configuration consumer, I expect Jakarta EE specifications like Persistence, Mail, Messaging, etc. to be configurable using property files as opposed to today's XML only configuration capability. This is a longer term expectation.
• As a configuration consumer, I expect to have a smooth migration from MicroProfile Config except for a namespace switch.

I hope this helps.

As I have stated previously, I would hope we can transition from MicroProfile Config with minimal changes. I have seen many customers use MicroProfile Config successfully. I understand this is an opportunity to re-examine things. However, if there is space for benefit of doubt, I would favor the MicroProfile Config pre-existing decisions.

Reza Rahman
Jakarta EE Ambassador, Author, Blogger, Speaker

Please note views expressed here are my own as an individual community member and do not reflect the views of my employer.

[dmlloyd]

Thanks Reza. I've captured these requirements, more or less, in the following cards:

• https://github.com/eclipse-ee4j/config/projects/1#card-75177887
• https://github.com/eclipse-ee4j/config/projects/1#card-75177939
• https://github.com/eclipse-ee4j/config/projects/1#card-75178084
• https://github.com/eclipse-ee4j/config/projects/1#card-75178168
• https://github.com/eclipse-ee4j/config/projects/1#card-75178273
• https://github.com/eclipse-ee4j/config/projects/1#card-75178458
• https://github.com/eclipse-ee4j/config/projects/1#card-75178563
• https://github.com/eclipse-ee4j/config/projects/1#card-75178655
• https://github.com/eclipse-ee4j/config/projects/1#card-75178765
• https://github.com/eclipse-ee4j/config/projects/1#card-75178837
• https://github.com/eclipse-ee4j/config/projects/1#card-75178917

I've reworded them somewhat and provided more specific role names (these are also subject to further revision so don't worry if something was missed in the first round). Also I've skipped a couple of items that specified expectations for implementation details, since we will be talking about that later rather than now; these can be recaptured at that time.

If you notice something that was left out or changed in a way that countermands your original intent, please let me know.

[m-reza-rahman]

Looks good for now I think. I hope this helps things move forward productively.

[jesse-gallagher]

I can second all of @m-reza-rahman 's expectations. I've used MicroProfile Config and enjoyed it thoroughly, and also wrote a custom contributor for a third-party framework I was using when writing my app. I'd also like to add:

• As a configuration consumer, I want to have a clear mechanism to know which provider "wins" if multiple sources provide the same key. For example, if an in-WAR .properties file has the same key as a system property and as a platform-specific extension source.
• As a configuration implementor, I want it to be that the priority system for the default sources is cleanly documented and easy to fully override or interleave between standard sources with a new one (e.g. they'd be priority 10, 20, 30 instead of 1, 2, 3 if using @Priority).

[dmlloyd]

Thanks Jesse. I've captured these requirements in the following cards:

• https://github.com/eclipse-ee4j/config/projects/1#card-75179211
• https://github.com/eclipse-ee4j/config/projects/1#card-75179371

I've also attempted to generalize the role names and language a bit but I think I kept the spirit of the requirements. If I have missed a point or countermanded your original intent in any way, please let me know.

[ljnelson]

Hi, @dmlloyd; thanks for getting this (re-) rolling. Here's what I came up with this morning following your brainstorm suggestion and in that spirit.

I guess you could call these the "raw materials" I personally would use to start theorizing about use cases (e.g. "component developer acquires environment object", "application assembler disambiguates conflicting paths", "application deployer deploys application into environment with coordinates"), but I haven't really gone all the way to full use cases yet (though I've written about related stuff before with no real engagement). I like to get my terms down and my roles down first. Unless otherwise noted, I made all of this up. Hopefully it's useful.

(In what follows I am as vendor- and implementation-neutral as possible to avoid accidentally locking people in to thinking about a particular implementation, because by definition one doesn't exist yet. Consequently you won't see, for example, Config, or ConfigSource, or "property", or "flat", or "tree", or "converter", or "conversion", and so on, below. That's all on purpose.)

Terms

In hopefully alphabetical order:

Application: A collection of assembled and disambiguated components that can be deployed.

Assemble: To put an application together out of components, resolving any ambiguities along the way, in preparation for its presumed deployment into an environment.

Component: A Java class, or a collection of Java classes, either notional or physical, packaged or not. What actually issues calls to
an implementation of this specification.

Configuration: An externalized, often persistent, portion of component state, or a snapshot of such a portion, that, when loaded as
a Java object into a component of a running application, may cause the component to change its behvior without there having to be any corresponding change in that component's source code.

Deploy: To start and run a (by definition assembled) application in an environment, or to set it up for pushbutton starting-and-running in an environment later.

Deserialize: To come up with a Java object from some sort of persistent representation.

Deserializer: A Java component that deserializes a Java object of one or more supported types, particularly in the service of an implementation of this specification.

Environment: (not crazy about the name) Configuration (see above) plus "automatic non-human stuff", like the current working directory, the current time, the machine architecture, the phase of the moon, the current locale, and so on. The notional "place" that a (by definition assembled) application is deployed into.

Environment Coordinates: Notional "latitude and longitude" identifiers that identify an environment. So-called "profiles" are a
crude version of this concept, but of course environment coordinates are not necessarily hierarchical.

Environment Object: (not happy with the name) A Java object retrieved by the Component Developer using an implementation of this specification.

Path: A notional "pointer" that logically picks out a portion of environmental state ("I don't want just any Frood, I want the 'hoopy' one"), but only partially, unless environment coordinates are also somehow supplied ("I don't want just any Frood, I want the "hoopy" one suitable for the environment this application is currently deployed into which is {checks notes} the test environment and the us-best region"). Consists of at least a name or other identifier and some indication of the (Java) type of thing it points to. The Component Developer (see below) identifies the Java object she wants to retrieve from configuration (or the environment) by using paths.

Pathspace: A notional universe within which any given path is unique. Each component has its own pathspace. An application has its own pathspace. Sometimes an application's components' pathspaces are compatible; sometimes they are not and require disambiguation.

Roles

In hopefully alphabetical order:

Application Assembler: Combines dependencies (components) together into some kind of (thereby) assembled application. (This is often played by a build tool, such as the maven-war-plugin, or a human being, such as was originally anticipated by the Java EE Platform Specification.) Has to reconcile components that may identify two different kinds of configuration objects with the same name, which is tricky because the components will have that name baked into their .class files, and their source code is not always available. Having performed this reconciliation, "advertises" the resulting configuration information in much the same way as the Component Developer (see below) so that the Configuration Author (see below) can create the right configuration information.

Application Deployer: Starts a (by definition assembled) application in an environment in such a way that an implementation of this specification used by the components of the application "knows" "where" it is (is it running in production? in the cloud? in test? with experimental features enabled? in the English locale? some of the above? exactly one of the above?). See also: Environment Coordinates above.

Component Developer: Develops Java libraries used as dependencies, either in "business code" or in, say, an application server's underpinnings; it doesn't matter. Writes code that retrieves Java objects from configuration, using (definitionally within the "pathspace" of her component) "paths" of some kind to designate which Java objects to retrieve. Does not know all the applications her components may be used in. Uses features of the configuration system and/or JDK (javap or something similar) to "advertise" the "paths" used to "find" Java objects from configuration so that the Configuration Author (see below) knows what configuration to write. Does not care at retrieval time about how the Java objects she retrieves are produced, stored, or deserialized; just cares that they're retrieved.

Configuration Author: Uses the configuration information advertised by the Application Assembler (see above) and/or the
Component Developer (see above) together with her knowledge of her organization's deployment environments and her organization's
configuration system (which may simply be directories and files, or may be something very complicated and bespoke) to "write" proper configuration for such environments and systems. Does not care at some level how it is retrieved or deserialized or by whom or when.

System Developer: (not happy with the name; maybe Infrastructure Developer) Adapts her organization's configuration system (which may simply be directories and files, or may be something very complicated and bespoke) to an implementation of this specification so that when a component "asks for" a Java object it can be successfully retrieved. Does not care who retrieves the Java object, but cares deeply (and controls) how it is deserialized from the configuration system being adapted.

[struberg]

I think one aspect was not yet mentioned: who is the target of using the configuration mechanism?

The initial goal when we started DeltaSpike Config was to give application programmers a way to tweak their own applications. It was not initially targetted at frameworks and containers itself!

The question was brought up by Joakim: is this approach also usable for containers themselves like Tomcat or Jetty? What do we need to adopt? What different approach might be better?