Map Loading/Unloading On-Demand

[jchamberlain]

Hello!

This ticket is trying to do two things: 1) consolidate some details of prior requests for dynamic configuration, 2) specify the requirements of my current project at GeoStrategies. I believe these two can go well together. I'm deliberately (mostly) staying away from implementation details in this first post so we can discuss just the requirements.

Previous/related issues: #57, #58, #262, #613, #933

Requirements

1. Load/Unload Maps On-demand with Zero Downtime

The primary objective is for the app to keep running and all existing maps to continue uninterrupted service while new maps are added and old maps are retired.

Since a map's layers must all exist in the provider, and layers may be created on-the-fly (SQL with various filters/transformations applied), this implies:

1-A. Load/Unload Providers On-demand

We require the ability to create new providers and remove old providers without downtime just as with maps. Critically, a provider's lifecycle must be tied in some way to the map's lifecycle, at the very least guaranteeing that the provider is created before and removed after any dependent maps.

1-B. Load New Map in <100ms

This is a rough guideline, but we want no perceivable delay between a user action triggering a map load and that map being available.

1-C. Load New Map Without Clearing Existing Connection Pools

Existing maps should continue service uninterrupted and undegraded. This means their providers can't be reinstantiated, i.e., we can't reload the whole configuration.

2. Synchronize Map Loading/Unloading Across Cluster

The method by which maps+providers are loaded/unloaded must not hinder horizontal scaling/HA of Tegola instances.

2-A. Use Shared Config

All instances in a cluster should be able to look at the same set of maps+providers. They should not require individual configuration, e.g., connecting to each instance and adding a config file.

2-B. Poll/Watch/Subscribe to Map/Providers Loads/Unloads

All instances in a cluster should be able to fetch or be notified of map+provider changes, keeping requirement I.B in mind (polling >10 times/sec to meet that goal is probably not ideal)

Note

All of this on-demand config change is limited to maps and providers. I see no need to make the remainder of the TOML file reloadable (at least in this manner). I see a strong distinction between general app/system config (like caching, observability, etc) which applies at boot time and maps which change constantly.

Note

I also see no need to modify existing maps or providers. For our purposes, we always add a new one and retire the old, with unique names being generated automatically.