diff --git a/blueprints/layered-profile-server.md b/blueprints/layered-profile-server.md new file mode 100644 index 0000000000..6b17b9d4db --- /dev/null +++ b/blueprints/layered-profile-server.md @@ -0,0 +1,238 @@ + +# Layered Profile + +## Problem Description + +Profiles are unwieldy and dangerous for Operations. + +Currently, we have countless Profiles, in broad categories. For example, "EDGE_123_FOO_ABC_ATS_714-RC0-42" might represent servers which are +1. Edges +2. Amiga 123 machines +3. In the Foo CDN +4. In the ABC datacenter +5. Running ATS 7.14 RC0 +6. Who knows what 42 means? + +Suppose we have Amiga 456 machines, at DEF datacenters, and a Bar CDN, and some servers are running ATS 7.15, 8.0, and 8.1. We make profiles for each server we have fitting all those categories: + +EDGE_123_FOO_ABC_ATS_714-RC0-27, EDGE_456_FOO_ABC_ATS_714-RC0-27, EDGE_123_BAR_ABC_ATS_714-RC0-42, EDGE_456_FOO_DEF_ATS_714-RC1-27, EDGE_123_FOO_DEF_ATS_800-RC4-29 + +- The number of Profiles quickly becomes a combinatorial explosion +- It's nearly impossible for Ops engineers to figure out what Parameters are assigned to all Profiles of a given CDN, machine, or datacenter. +- What about that one random Parameter on a Cloned profile that was changed a year ago? Is it still in place? Should it be? Did it need to be applied to all new Profiles cloned from this one? + +## Proposed Change + +Layered Profiles allow assigning multiple profiles to Servers. If multiple profiles have a parameter with the same name and config file, the parameter from the last profile in the ordering is applied. + +Layered Profiles is exactly as powerful as the existing profiles, it doesn't enable any new things. It makes profiles much easier to manage. + +With Layered Profiles, hundreds of profiles become a few dozen, each representing a logical group. For example, a server might have the profiles, in order: +1. EDGE +2. AMIGA_123 +3. CDN_FOO +4. DATACENTER_ABC +5. ATS_714 +6. Custom_Hack_42 + +### Traffic Portal Impact + +1. A UI to view all parameters currently applied to a server, as well as the profile each parameter came from. A new page, linked from Server pages. For eg: + +| Name |Config File | Value | Profile | +|--------------------------------------------------|--------------------------|--------------------------------------|-----------| +| location | url_sig_myds.config | /opt/trafficserver/etc/trafficserver | EDGE | +| Drive_Prefix | storage.config | /dev/sd | AMIGA_123 | +| error_url | url_sig_myotherds.config | 403 | EDGE | +| CONFIG proxy.config.exec_thread.autoconfig.scale | records.config | FLOAT 1.5 | ATS_714 | + +2. A UI change to add, remove, and reorder (sortable list) profiles for Servers, on the existing Server pages. +3. Filtering based on Profiles will also need to be updated to take into account the plurality of Profiles. + +### Traffic Ops Impact + +- Add backend logic to the below-mentioned existing API endpoints, in order to show/create/update/delete a sorted list of profiles for server. + +- `/servers/ GET, POST` +- `/servers/{id} PUT, DELETE` +- `/servers/details` +- `/deliveryservices/{{ID}}/servers` +- `/deliveryservices/{{ID}}/servers/eligible` + +#### REST API Impact + +- No new endpoints are required + +**Existing Endpoints** + +- Modify JSON request and response for existing servers endpoints. +- **_Note_**: All fields in the response structure for API 4.0 except `profileId`, `profileDesc` and `profile`, remain same. + - In API 4.0 `profile` changes to `profiles` and `profileId` and `profileDesc` are no longer part of response structure + +#### API 4.0 GET +- JSON **response** with the proposed change will look as follows: + `/servers?id=5` +```JSON +{ + "response": [{ + "id": 5, + "profiles": ["MID", "AMIGA_123", "CDN_FOO"] + }] +} +``` + +`/servers` +```JSON +{ + "response": [{ + "profiles": ["MID", "AMIGA_123", "CDN_FOO"] + }] +} +``` + +#### API 4.0 POST/PUT +JSON **request** with the proposed change will look as follows: + +`POST /servers ` +```JSON +{ + "cachegroupId": 6, + "cdnId": 2, + "profiles": ["MID", "AMIGA_123", "CDN_FOO"] +} +``` + +`PUT /servers/5` +```JSON +{ + "cachegroupId": 6, + "cdnId": 2, + "profiles": ["MID", "AMIGA_123", "CDN_FOO"] +} +``` + +return following **response** +```JSON +{ "alerts": [ + { + "text": "Server created/updated", + "level": "success" + } +], + "response": { + "cachegroup": "CDN_in_a_Box_Mid", + "cachegroupId": 6, + "cdnId": 2, + "id": 5, + "profiles": ["MID", "AMIGA_123", "CDN_FOO"] + } +} +``` + +The following table describes the top level `layered_profile` object for servers: + +| field | type | optionality | description | +| ----------------| ---------------------| ----------- | ---------------------------------------------------------| +| server | bigint | required | the server id associated with a given profile | +| profiles | text [] | required | the profile names associated with a server | +| order | bigint | required | the order in which a profile is applied to a server | + +**API constraints** +- In API 4.0, + - GET `/servers` object, `profiles` field is an array, + - and a PUT or POST `/servers` allows multiple profiles to be assigned, and displayed in the UI, etc. + The UI may or may not display a list (which can only add 1), but the client implements handling multiple, and the API is documented to potentially return multiple and how their parameters must be applied. +- Add `Profiles` key to API endpoints for Server objects + +#### Client Impact +- Existing Go client methods will be updated for the `/servers` endpoints in order to write TO API tests for the exising endpoints. +- All functions which get Parameters on Servers must be changed to get the Parameters of all assigned Profiles in order. + +#### Data Model / Database Impact + +- A new Database Table `server_profiles` as described below, will be created: +```text + Table "traffic_ops.server_profiles" + Column | Type | Collation | Nullable | Default +---------------+--------------------------+-----------+----------+-------- + server | bigint | | not null | + profile_name | text | | not null | + order | bigint | | not null | +Indexes: + "pk_server_profile" PRIMARY KEY(profile_name, server, order) +Foreign-key constraints: + "fk_server" FOREIGN KEY (server) REFERENCES server(id) + "fk_server" FOREIGN KEY (profile_name) REFERENCES profile(name) +``` + +All profiles assigned to a given server will have the same values of: +- type +- cdn +If any of those differ within the same server, it's probably a mistake. + +### ORT Impact +New set of profiles will be created and the profile-parameter relationship will change. + +### Traffic Monitor Impact +No impact + +### Traffic Router Impact +These changes will affect the Snapshot generation (both crconfig and monitoring). Even though that is more of a TO impact. +Reason being that Snapshots and Monitoring Configurations for a CDN include Profile and Parameter information for the servers, Traffic Monitors, and Traffic Routersz. + +### Traffic Stats Impact +No impact + +### Traffic Vault Impact +No impact + +### Documentation Impact +All existing endpoints will need to be updated, along with the documentation explaining `layered_profile`. + +### Testing Impact +Client/API integration tests should be updated to verify the `layered_profile` functionality on existing API `/servers` endpoints. + +### Performance Impact +We do not anticipate any performance impact with layered_profile approach. + +### Security Impact +We do not anticipate any impact on security. + +### Upgrade Impact +- A Database Migration to: + - drop profile column in existing server table + - insert existing server profiles along with their order into the new table(server_profiles) + +### Operations Impact +The profile-parameter relationship will change based on new sets of profile and Operations will have to learn on how to assign profile order to a server. + +### Developer Impact +Developers will most likely need to use layered_profile, so they'll need to be familiar with the process +of adding, sorting, deleting, debugging and working with layered_profiles. +When searching for any parameters assigned to a profile for a given server, one will need to look up all profiles assigned to the server and then process all the parameters in order to get the "final" view of the profile. + +## Alternatives +None, except to keep using existing Profiles. + +## Dependencies +None + +## References +None