Epic: support multiple charting libraries via "engines"

[tomwayson]

Cedar v0 was written on top of vega/d3 and cedar v1 will be written on top of amCharts, yet, outside of override we've been able to keep much of the same API. We should in theory be able to be able isolate the implementation details specific to a given charting library into an "engine" that the consuming library can load depending on their needs.

API

The likely API would look like:

const config = {
    engine: 'vega', // defaults to 'amCharts'?
    type: 'bar'
    ...
};
const chart = new Cedar(config);
chart.show("chartDiv"); // show() calls the appropriate render method from Cedar.vega or throws an error if that has not been loaded

In addition, we may want to support:

• using more than one engine in a single app on a chart by chart basis
• dynamically loading the engine code at run time

Packages, dependencies, and loading

The challenge is how do we allow consuming applications to only load the engine(s) that they need in their app.

Option 1: The "Plugin" Model

The first options is that we each keep the current idea that engine is a separate package, but we invert the current dependency structure (where cedar depends on cedar-amcharts) so that each engine would depend on cedar (likely as a peer). The base (cedar) package would define a global namespace (Cedar), and any engines are also loaded would append themselves to that namespace (Cedar.amCharts, Cedar.vega), similar to the way that Leaflet and amCharts plugins work.

When using the UMD build:

<!-- in this case we're using the CDNs, but these scripts could point to local files instead -->
<!-- 
  first load the external charting library
  for v1 this will always be amCharts, but in the future
  it would be based on which engine(s) the app will use
--> 
<script src="https://www.amcharts.com/lib/3/amcharts.js"></script>
<script src="https://www.amcharts.com/lib/3/serial.js"></script>
<!-- now load the cedar base library -->
<script src="https://unpkg.com/@esri/cedar/dist/cedar.js"></script>
<!-- now load which ever engine you are using, for v1 this is always amCharts -->
<script src="https://unpkg.com/@esri-amcharts/cedar/dist/cedar-amcharts.js"></script>

Or if installing from the registry for use in a custom local build:

# NOTE: consuming apps can chose whether or not to treat amCharts as external
npm i amcharts3 @esri/cedar @esri/cedar-amcharts

Then you would include these lines in the consuming app:

import AmCharts from 'amcharts3'
import Cedar from '@esri/cedar';
// NOTE: only importing this for the side effect of appending to `Cedar.amCharts`
import '@esri/cedar-amcharts'

PROs:

• established pattern used by other libraries, well understood

CONs:

• consumers need to know what engine(s) they need and understand the dependencies

Also, not really a CON, but to me it is somewhat implicit that you can use the base package (cedar) w/o the plugin (like Leaflet), but in our case you can't. This is OK, I think it's how amCharts works (i.e. you need to add serial or xy to do anything).

Option 2 One Cedar to Rule Them All

We could get rid of cedar-amcharts package and include its code within the cedar package. Future engines like cedar-vega would be added inside the cedar package as well. We'd then rely on something like dynamic import() statements to load whichever engine(s) are needed at runtime.

<!-- in this case we're using the CDNs, but these scripts could point to local files instead -->
<!-- 
  first load the external charting library
  for v1 this will always be amCharts, but in the future
  it would be based on which engine(s) the app will use
--> 
<script src="https://www.amcharts.com/lib/3/amcharts.js"></script>
<script src="https://www.amcharts.com/lib/3/serial.js"></script>
<!-- 
  then just load cedar
-->
<script src="https://unpkg.com/@esri/cedar/dist/cedar.js"></script>

Or if installing from the registry for use in a local build:

# NOTE: consuming apps can chose whether or not to treat amCharts as external
npm i amcharts3 @esri/cedar

Then in the consuming app's code:

import AmCharts from 'amcharts3'
import Cedar from '@esri/cedar'

PROS:

• consuming apps don't need to know about or explicitly load engines, cedar handles this for them

CONS:

• While I'm pretty sure we can get dynamic import() statements or something similar to work in a CDN where we have full control over the build pipeline and distributed artifacts, I'm not sure we'd be able to set those up in a way that would work with every consuming app's build toolchain. We might even need to change our own toolchain just to get it working in our apps. While TypeScript now supports dynamic import(), unfortunately rollup does not yet and nor does ember-cli. Maybe we'd have to swtich out rollup for webpack 🤦‍♂️, or maybe drop rollup and just have TypeScript to generate the UMD build (since it's all one package) and maybe, just maybe whatever it spits out will actually work in Ember after it's been equirerayed by one of the Esri addons... maybe.

If for whatever reason we really felt like we needed to have each engine in it's own package that would only complicate the above issues even further. So if separate packages is a requirement, I'd suggest we look into the Option 1 instead.

I hate to admit this, but Dojo 1's AMD build with staticHasFeatures is the perfect solution to this problem. I'm somewhat encouraged by the fact that Dojo2 is ditching their own dedicated loader now that import() has landed in TS and curious if/how they plan to make something like staticHasFeatures work w/ import() and webpack... but I digress. My point is, I think this idea is the way of the future, but it's not the future yet.

[ajturner]

I prefer option 1

For loading dependencies, can engines do this themselves at load time? Inject script tags for libraries?

[tomwayson]

Thanks for weighing in @ajturner.

For loading dependencies, can engines do this themselves at load time? Inject script tags for libraries?

They can. I'm not totally convinced that they should, or at least not as the default behavior. What I'd suggest is that we start w/ forcing consumers to bring their own charting library (from CDN, a local copy, or bundled into their application's build, whatever suits the specific needs of their app). Then we can consider adding a feature where the engine detects if the required library is not present, and if not it tries to load it. The question is from where? The CDN? A copy distributed w/ Cedar? I really don't want to get into that business again. Can the user provide a url to override where Cedar tries to pull it from? But if they know they need the library, and they know where they want to get it from, why didn't they just get it themselves?