[Vis Colors] Update color mapper to prioritize unique colors per vis

[joshuarrrr]

Description

Instead of trying to generate unique colors for an entire dashboard.

Visualizations that use the color mapper (VisLib, VisBuilder, TSVB) will now have the following behavior:

1. Every visualization will preferentially use the colors from the OUI qualitative palette. Only if there are more than 10 colors needed for a single visualization, additional colors will be generated via rotations of that palette. (new feature)
2. Mapped colors on dashboards are no longer subject to race conditions, where multiple loads of the same dashboard would result in different mapped colors. That means dashboard colors are stable and there's no need to manually specify colors just to achieve consistency. (new feature)
3. Series labels that appear in multiple visualization in the same dashboard (e.g. status code "404") will have the same color, even if they're ordered differently. (existing feature maintained)
4. Within an individual visualization, every data series will get a unique color with no repeats (existing feature maintained)
5. Custom color configurations specified via the visualization:colorMapping advanced setting will be respected. Additionally, if one or more series in a visualization have custom color configurations that use OUI qualitative palette colors, those will be skipped when mapping non-customized colors to avoid duplicates. (new feature)
6. If users manually set colors via the legend (either in visualization or dashboard), it will not affect the color mapping (those are treated as overrides to mapped colors). (existing feature maintained)

Issues Resolved

Fixes #4697

Screenshot

Sample dashboards (next light) after

Notice the consistency across dashboards and data sets:

Sample dashboards (next light) before

Visual Consistency Dashboard (all themes) after

Testing the changes

Check List

• All tests pass
  • yarn test:jest
  • yarn test:jest_integration
  • yarn test:ftr
• New functionality includes testing.
• New functionality has been documented.
• Update CHANGELOG.md
• Commits are signed per the DCO using --signoff

[codecov]

Codecov Report

Merging #4890 (6c5b497) into main (ce2d79e) will decrease coverage by 0.06%.
Report is 4 commits behind head on main.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #4890      +/-   ##
==========================================
- Coverage   66.39%   66.34%   -0.06%     
==========================================
  Files        3396     3397       +1     
  Lines       64801    64805       +4     
  Branches    10359    10360       +1     
==========================================
- Hits        43025    42994      -31     
- Misses      19217    19222       +5     
- Partials     2559     2589      +30     

Flag	Coverage Δ
Linux_1	?
Linux_2	?
Linux_3	?
Linux_4	?
Windows_1	34.86% <33.33%> (-0.01%)	⬇️
Windows_2	55.12% <ø> (ø)
Windows_3	44.24% <100.00%> (+<0.01%)	⬆️
Windows_4	34.92% <0.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
...rc/plugins/charts/public/services/colors/colors.ts	100.00% <ø> (ø)
...ins/charts/public/services/colors/mapped_colors.ts	96.42% <100.00%> (+3.83%)	⬆️

... and 23 files with indirect coverage changes

[AMoo-Miki]

Do we need to check that after filtering, we have enough colors? as much as keysToMap.length? or we don't need to worry about it?

[joshuarrrr]

@AMoo-Miki I made this change as more of a surgical minimal change rather than a larger refactor, which this utility could probably use.

My intent is that it will always be the correct length, because we're using the total keys.length in L110 to generate the colors, and then we're only filtering out alreadyUsedColors. The intent of the L88-106 block is to push every key in keys to either alreadyUsedColors or keysToMap. But it could certainly be written cleaner to guarantee that's the case.

[kavilla]

Thank you!