Improve TQ Stats docs

openapi/openapiv2.json

@@ -10776,24 +10776,24 @@
         "approximateBacklogCount": {
           "type": "string",
           "format": "int64",
-          "description": "The approximate number of tasks backlogged in this task queue. May count expired tasks but eventually converges\nto the right value."
+          "description": "The approximate number of tasks backlogged in this task queue. May count expired tasks but eventually\nconverges to the right value. Can be relied upon for scaling decisions.\n\nSpecial note for workflow task queue type: this metric does not count sticky queue tasks. However, because\nthose tasks only remain valid for a few seconds, the inaccuracy becomes less significant as the backlog size\ngrows."
         },
         "approximateBacklogAge": {
           "type": "string",
-          "description": "Approximate age of the oldest task in the backlog based on the create timestamp of the task at the head of the queue."
+          "description": "Approximate age of the oldest task in the backlog based on the creation time of the task at the head of\nthe queue. Can be relied upon for scaling decisions.\n\nSpecial note for workflow task queue type: this metric does not count sticky queue tasks. However, because\nthose tasks only remain valid for a few seconds, they should not affect the result when backlog is older than\nfew seconds."
         },
         "tasksAddRate": {
           "type": "number",
           "format": "float",
-          "description": "Approximate tasks per second added to the task queue based on activity within a fixed window. This includes both backlogged and\nsync-matched tasks."
+          "description": "The approximate tasks per second added to the task queue, averaging the last 30 seconds. These includes tasks\nwhether or not they were added to/dispatched from the backlog or they were dispatched immediately without going\nto the backlog (sync-matched).\n\nThe difference between `tasks_add_rate` and `tasks_dispatch_rate` is a reliable metric for the rate at which\nbacklog grows/shrinks. See `BacklogIncreaseRate`.\n\nNote: the actual tasks delivered to the workers may significantly higher than the numbers reported by\ntasks_add_rate, because:\n- Tasks can be sent to workers without going to the task queue. This is called Eager dispatch. Eager dispatch is\n  enable for activities by default in the latest SDKs.\n- Tasks going to Sticky queue are not accounted for. Note that, typically, only the first workflow task of each\n  workflow goes to a normal queue, and the rest workflow tasks go to the Sticky queue associated with a specific\n  worker instance."
         },
         "tasksDispatchRate": {
           "type": "number",
           "format": "float",
-          "description": "Approximate tasks per second dispatched to workers based on activity within a fixed window. This includes both backlogged and\nsync-matched tasks."
+          "description": "The approximate tasks per second dispatched from the task queue, averaging the last 30 seconds. These includes\ntasks whether or not they were added to/dispatched from the backlog or they were dispatched immediately without\ngoing to the backlog (sync-matched).\n\nThe difference between `tasks_add_rate` and `tasks_dispatch_rate` is a reliable metric for the rate at which\nbacklog grows/shrinks. See `BacklogIncreaseRate`.\n\nNote: the actual tasks delivered to the workers may significantly higher than the numbers reported by\ntasks_dispatch_rate, because:\n- Tasks can be sent to workers without going to the task queue. This is called Eager dispatch. Eager dispatch is\n  enable for activities by default in the latest SDKs.\n- Tasks going to Sticky queue are not accounted for. Note that, typically, only the first workflow task of each\n  workflow goes to a normal queue, and the rest workflow tasks go to the Sticky queue associated with a specific\n  worker instance."
         }
       },
-      "description": "For workflow task queues, we only report the normal queue stats, not sticky queues. This means the stats\nreported here do not count all workflow tasks. However, because the tasks queued in sticky queues only remain\nvalid for a few seconds, the inaccuracy becomes less significant as the backlog age grows."
+      "description": "TaskQueueStats contains statistics about task queue backlog and activity.\n\nFor workflow task queue type, this result is partial because tasks sent to sticky queues are not included. Read\ncomments above each metric to understand the impact of sticky queue exclusion on that metric accuracy."
     },
     "v1TaskQueueStatus": {
       "type": "object",

openapi/openapiv3.yaml

@@ -8231,28 +8231,63 @@ components:
         approximateBacklogCount:
           type: string
           description: |-
-            The approximate number of tasks backlogged in this task queue. May count expired tasks but eventually converges
-             to the right value.
+            The approximate number of tasks backlogged in this task queue. May count expired tasks but eventually
+             converges to the right value. Can be relied upon for scaling decisions.
+
+             Special note for workflow task queue type: this metric does not count sticky queue tasks. However, because
+             those tasks only remain valid for a few seconds, the inaccuracy becomes less significant as the backlog size
+             grows.
         approximateBacklogAge:
           pattern: ^-?(?:0|[1-9][0-9]{0,11})(?:\.[0-9]{1,9})?s$
           type: string
-          description: Approximate age of the oldest task in the backlog based on the create timestamp of the task at the head of the queue.
+          description: |-
+            Approximate age of the oldest task in the backlog based on the creation time of the task at the head of
+             the queue. Can be relied upon for scaling decisions.
+
+             Special note for workflow task queue type: this metric does not count sticky queue tasks. However, because
+             those tasks only remain valid for a few seconds, they should not affect the result when backlog is older than
+             few seconds.
         tasksAddRate:
           type: number
           description: |-
-            Approximate tasks per second added to the task queue based on activity within a fixed window. This includes both backlogged and
-             sync-matched tasks.
+            The approximate tasks per second added to the task queue, averaging the last 30 seconds. These includes tasks
+             whether or not they were added to/dispatched from the backlog or they were dispatched immediately without going
+             to the backlog (sync-matched).
+
+             The difference between `tasks_add_rate` and `tasks_dispatch_rate` is a reliable metric for the rate at which
+             backlog grows/shrinks. See `BacklogIncreaseRate`.
+
+             Note: the actual tasks delivered to the workers may significantly higher than the numbers reported by
+             tasks_add_rate, because:
+             - Tasks can be sent to workers without going to the task queue. This is called Eager dispatch. Eager dispatch is
+               enable for activities by default in the latest SDKs.
+             - Tasks going to Sticky queue are not accounted for. Note that, typically, only the first workflow task of each
+               workflow goes to a normal queue, and the rest workflow tasks go to the Sticky queue associated with a specific
+               worker instance.
           format: float
         tasksDispatchRate:
           type: number
           description: |-
-            Approximate tasks per second dispatched to workers based on activity within a fixed window. This includes both backlogged and
-             sync-matched tasks.
+            The approximate tasks per second dispatched from the task queue, averaging the last 30 seconds. These includes
+             tasks whether or not they were added to/dispatched from the backlog or they were dispatched immediately without
+             going to the backlog (sync-matched).
+
+             The difference between `tasks_add_rate` and `tasks_dispatch_rate` is a reliable metric for the rate at which
+             backlog grows/shrinks. See `BacklogIncreaseRate`.
+
+             Note: the actual tasks delivered to the workers may significantly higher than the numbers reported by
+             tasks_dispatch_rate, because:
+             - Tasks can be sent to workers without going to the task queue. This is called Eager dispatch. Eager dispatch is
+               enable for activities by default in the latest SDKs.
+             - Tasks going to Sticky queue are not accounted for. Note that, typically, only the first workflow task of each
+               workflow goes to a normal queue, and the rest workflow tasks go to the Sticky queue associated with a specific
+               worker instance.
           format: float
       description: |-
-        For workflow task queues, we only report the normal queue stats, not sticky queues. This means the stats
-         reported here do not count all workflow tasks. However, because the tasks queued in sticky queues only remain
-         valid for a few seconds, the inaccuracy becomes less significant as the backlog age grows.
+        TaskQueueStats contains statistics about task queue backlog and activity.
+
+         For workflow task queue type, this result is partial because tasks sent to sticky queues are not included. Read
+         comments above each metric to understand the impact of sticky queue exclusion on that metric accuracy.
     TaskQueueStatus:
       type: object
       properties:

temporal/api/taskqueue/v1/message.proto

@@ -87,20 +87,54 @@ message TaskQueueTypeInfo {
     TaskQueueStats stats = 2;
 }
 
-// For workflow task queues, we only report the normal queue stats, not sticky queues. This means the stats
-// reported here do not count all workflow tasks. However, because the tasks queued in sticky queues only remain
-// valid for a few seconds, the inaccuracy becomes less significant as the backlog age grows.
+// TaskQueueStats contains statistics about task queue backlog and activity.
+//
+// For workflow task queue type, this result is partial because tasks sent to sticky queues are not included. Read
+// comments above each metric to understand the impact of sticky queue exclusion on that metric accuracy.
 message TaskQueueStats {
-    // The approximate number of tasks backlogged in this task queue. May count expired tasks but eventually converges
-    // to the right value.
+    // The approximate number of tasks backlogged in this task queue. May count expired tasks but eventually
+    // converges to the right value. Can be relied upon for scaling decisions.
+    //
+    // Special note for workflow task queue type: this metric does not count sticky queue tasks. However, because
+    // those tasks only remain valid for a few seconds, the inaccuracy becomes less significant as the backlog size
+    // grows.
     int64 approximate_backlog_count = 1;
-    // Approximate age of the oldest task in the backlog based on the create timestamp of the task at the head of the queue.
+    // Approximate age of the oldest task in the backlog based on the creation time of the task at the head of
+    // the queue. Can be relied upon for scaling decisions.
+    //
+    // Special note for workflow task queue type: this metric does not count sticky queue tasks. However, because
+    // those tasks only remain valid for a few seconds, they should not affect the result when backlog is older than
+    // few seconds.
     google.protobuf.Duration approximate_backlog_age = 2;
-    // Approximate tasks per second added to the task queue based on activity within a fixed window. This includes both backlogged and
-    // sync-matched tasks.
+    // The approximate tasks per second added to the task queue, averaging the last 30 seconds. These includes tasks
+    // whether or not they were added to/dispatched from the backlog or they were dispatched immediately without going
+    // to the backlog (sync-matched).
+    //
+    // The difference between `tasks_add_rate` and `tasks_dispatch_rate` is a reliable metric for the rate at which
+    // backlog grows/shrinks.
+    //
+    // Note: the actual tasks delivered to the workers may significantly be higher than the numbers reported by
+    // tasks_add_rate, because:
+    // - Tasks can be sent to workers without going to the task queue. This is called Eager dispatch. Eager dispatch is
+    //   enable for activities by default in the latest SDKs.
+    // - Tasks going to Sticky queue are not accounted for. Note that, typically, only the first workflow task of each
+    //   workflow goes to a normal queue, and the rest workflow tasks go to the Sticky queue associated with a specific
+    //   worker instance.
     float tasks_add_rate = 3;
-    // Approximate tasks per second dispatched to workers based on activity within a fixed window. This includes both backlogged and
-    // sync-matched tasks.
+    // The approximate tasks per second dispatched from the task queue, averaging the last 30 seconds. These includes
+    // tasks whether or not they were added to/dispatched from the backlog or they were dispatched immediately without
+    // going to the backlog (sync-matched).
+    //
+    // The difference between `tasks_add_rate` and `tasks_dispatch_rate` is a reliable metric for the rate at which
+    // backlog grows/shrinks.
+    //
+    // Note: the actual tasks delivered to the workers may significantly be higher than the numbers reported by
+    // tasks_dispatch_rate, because:
+    // - Tasks can be sent to workers without going to the task queue. This is called Eager dispatch. Eager dispatch is
+    //   enable for activities by default in the latest SDKs.
+    // - Tasks going to Sticky queue are not accounted for. Note that, typically, only the first workflow task of each
+    //   workflow goes to a normal queue, and the rest workflow tasks go to the Sticky queue associated with a specific
+    //   worker instance.
     float tasks_dispatch_rate = 4;
 }