[WIP][SWIP-8] Support Kong monitoring

[CodePrometheus]

Motivation

Kong or Kong API Gateway is a cloud-native, platform-agnostic, scalable API Gateway distinguished for its high performance and extensibility via plugins. Now I want to add Kong monitoring via the OpenTelemetry Collector, which fetches metrics from it's own HTTP endpoint to expose metrics data for Prometheus.

Architecture Graph

There is no significant architecture-level change.

Proposed Changes¹

1. Kong expose its own metrics via HTTP endpoint to opentelemetry collector², OpenTelemetry Collector fetches metrics from it and pushes metrics to SkyWalking OTEL Receiver via OpenTelemetry exporter.
2. The SkyWalking OAP Server parses the expression with MAL to filter/calculate/aggregate and store the results.
3. These metrics can be displayed via the SkyWalking UI, and the metrics can be customized for display on the UI dashboard.

Counter

Monitoring Panel	Unit	Metric Name	Description	Data Source
Bandwidth	bytes	meter_kong_service_ingress_bandwidth_bytes meter_kong_instance_ingress_bandwidth_bytes meter_kong_endpoint_ingress_bandwidth_bytes	Total bandwidth ingress throughput in bytes	Kong
Bandwidth	bytes	meter_kong_service_egress_bandwidth_bytes meter_kong_instance_egress_bandwidth_bytes meter_kong_endpoint_egress_bandwidth_bytes	Total bandwidth egress throughput in bytes	Kong
HTTP	count	meter_kong_service_http_status meter_kong_instance_http_status meter_kong_endpoint_http_status	HTTP status codes per consumer/service/route in Kong	Kong
Nginx	count	meter_kong_service_kong_nginx_metric_errors_total meter_kong_instance_kong_nginx_metric_errors_total meter_kong_endpoint_kong_nginx_metric_errors_total meter_kong_node_kong_nginx_metric_errors_total	Number of nginx-lua-prometheus errors	Kong

Gauge

Monitoring Panel	Unit	Metric Name	Description	Data Source
DB	count	meter_kong_service_kong_datastore_reachable meter_kong_instance_kong_datastore_reachable	Datastore reachable from Kong	Kong
HTTP	count	meter_kong_service_http_requests meter_kong_instance_http_requests meter_kong_node_http_requests	Total number of requests	Kong
DB	bytes	meter_kong_service_kong_core_db_cache_miss_allocated_bytes meter_kong_instance_kong_core_db_cache_miss_allocated_bytes meter_kong_node_kong_core_db_cache_miss_allocated_bytes	Allocated slabs in bytes in a shared_dict about kong_core_db_cache_miss	Kong
DB	bytes	meter_kong_service_kong_core_db_cache_miss_total_bytes meter_kong_instance_kong_core_db_cache_miss_total_bytes meter_kong_node_kong_core_db_cache_miss_total_bytes	Total capacity in bytes of a shared_dict about kong_core_db_cache_miss	Kong
DB	bytes	meter_kong_service_kong_memory_workers_lua_vms_bytes meter_kong_instance_kong_memory_workers_lua_vms_bytes meter_kong_node_kong_memory_workers_lua_vms_bytes	Allocated bytes in worker Lua VM	Kong
Connections	count	meter_kong_service_kong_nginx_connections_total meter_kong_instance_kong_nginx_connections_total meter_kong_node_kong_nginx_connections_total	Number of connections by subsystem	Kong
Nginx	count	meter_kong_service_kong_nginx_timers meter_kong_instance_kong_nginx_timers	Number of Nginx timers	Kong

Histogram

Monitoring Panel	Unit	Metric Name	Description	Data Source
Latency	ms	meter_kong_service_kong_latency meter_kong_instance_kong_latency meter_kong_endpoint_kong_latency	Latency added by Kong and enabled plugins for each service/route in Kong	Kong
Latency	ms	meter_kong_service_kong_request_latency meter_kong_instance_kong_request_latency meter_kong_endpoint_kong_request_latency	Total latency incurred during requests for each service/route in Kong	Kong
Latency	ms	meter_kong_service_kong_upstream_latency meter_kong_instance_kong_upstream_latency meter_kong_endpoint_kong_upstream_latency	Latency added by upstream response for each service/route in Kong	Kong

Imported Dependencies libs and their licenses.

No new dependency.

Compatibility

No breaking changes.

General usage docs

References

1. Other SWIPs
2. Kong Docs

[wu-sheng]

I think we are good to have this.

[wu-sheng]

You could update the next swip ID first, and merge the swip-8 doc later with the feature.

[wu-sheng]

@CodePrometheus We are going to release 10.1 just before the National Day or after that holiday. Could you share are you going to add this into 10.1 or later?

[CodePrometheus]

I really hope it will be 10.1, but at the current pace, it's highly likely that I won't be able to catch up before the holiday.

[wu-sheng]

Thanks for the update. We will see whether we are possible to get there. There is one more feature of BanyanDB 0.7 integration.

[CodePrometheus]

As the feature progressed, I find that the metrics in the first version could has a better way of handling and categorized, I will make some adjustments to the SWIP and submit a PR when I'm ready.

[wu-sheng]

OK. Take your time. I will initial the 10.1 release before Oct 1st, I think. You could put this in 10.2.

[wu-sheng]

From what I saw from your screenshot, I would like to ask, would you have route rule metrics? I could set them as endpoint.

[CodePrometheus]

Yes, I divide them into service (cluster), instance and endpoint (route).