Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add client side request statistics to query diagnostics #877

Merged
merged 16 commits into from
Oct 23, 2019

Conversation

j82w
Copy link
Contributor

@j82w j82w commented Oct 7, 2019

Pull Request Template

Description

  1. Added the PointOperationStatistics to the query diagnostics. This allows the user to log all the information. Previously all the PointOperationStatistics information was not returned to the user.
  2. Added a new QueryPageDiagnostics to represent a single response from the backend and all the related telemetry.
  3. The QueryPageDiagnostics does not parse any of the information. This allows it to be lazy so it is only parsed when the user request the diagnostics information.
  4. QueryPageDiagnostics does not have any reference to CosmosDiagnostics to avoid leaking the v3 types for the query core logic.
  5. Query always returns a diagnostic now so users do not have to check for null. It returns an empty JSON array as the string.

Example:

[
    {
        "PartitionKeyRangeId": "0",
        "QueryMetricText": "totalExecutionTimeInMs=0.34;queryCompileTimeInMs=0.05;queryLogicalPlanBuildTimeInMs=0.03;queryPhysicalPlanBuildTimeInMs=0.00;queryOptimizationTimeInMs=0.00;VMExecutionTimeInMs=0.02;indexLookupTimeInMs=0.00;documentLoadTimeInMs=0.01;systemFunctionExecuteTimeInMs=0.00;userFunctionExecuteTimeInMs=0.00;retrievedDocumentCount=2;retrievedDocumentSize=1358;outputDocumentCount=1;outputDocumentSize=728;writeOutputTimeInMs=0.00;indexUtilizationRatio=0.50",
        "IndexUtilizationText": "",
        "RequestDiagnostics": {
            "ActivityId": "fda9aed0-2330-4b9f-a449-93a9372e2f1e",
            "StatusCode": 200,
            "SubStatusCode": 0,
            "RequestCharge": 2.3,
            "Method": {
                "Method": "POST"
            },
            "RequestUri": "dbs/f8d860b7-d1e7-4fa5-ae84-16461b85f5dd/colls/b8a6fab9-2ec8-4294-aa84-56764eaccf22",
            "requestStartTime": "2019-10-18T18:18:04.3683558Z",
            "requestEndTime": "2019-10-18T18:18:04.3751475Z",
            "responseStatisticsList": [
                {
                    "RequestResponseTime": "2019-10-18T18:18:04.3751475Z",
                    "StoreResult": {
                        "LSN": 2447,
                        "PartitionKeyRangeId": "0",
                        "QuorumAckedLSN": 2447,
                        "GlobalCommittedLSN": -1,
                        "NumberOfReadRegions": 0,
                        "ItemLSN": -1,
                        "SessionToken": {
                            "LSN": 2447
                        },
                        "UsingLocalLSN": true,
                        "RequestCharge": 2.3,
                        "CurrentReplicaSetSize": 1,
                        "CurrentWriteQuorum": 1,
                        "IsValid": true,
                        "IsGoneException": false,
                        "IsNotFoundException": false,
                        "IsInvalidPartitionException": false,
                        "StorePhysicalAddress": "rntbd://127.0.0.1:10253/apps/DocDbApp/services/DocDbServer9/partitions/a4cb4955-38c8-11e6-8106-8cdcd42c33be/replicas/1p/",
                        "IsClientCpuOverloaded": false
                    },
                    "RequestResourceType": 2,
                    "RequestOperationType": 15
                }
            ],
            "supplementalResponseStatisticsList": [],
            "addressResolutionStatistics": {},
            "contactedReplicas": [
                "rntbd://127.0.0.1:10253/apps/DocDbApp/services/DocDbServer9/partitions/a4cb4955-38c8-11e6-8106-8cdcd42c33be/replicas/1p/"
            ],
            "failedReplicas": [],
            "regionsContacted": [
                "https://127.0.0.1:8081/"
            ],
            "requestLatency": "00:00:00.0067917"
        },
        "SchedulingTimeSpan": {
            "TurnaroundTimeInMs": "78.7925",
            "ResponseTimeInMs": "18.8618",
            "RunTimeInMs": "48.0142",
            "WaitTimeInMs": "30.7785",
            "NumberOfPreemptions": "2"
        }
    }
]

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

@j82w j82w added the QUERY label Oct 7, 2019
@j82w j82w self-assigned this Oct 7, 2019
@@ -153,15 +148,14 @@ internal sealed class ItemProducer

this.fetchSchedulingMetrics = new SchedulingStopwatch();
this.fetchSchedulingMetrics.Ready();
this.fetchExecutionRangeAccumulator = new FetchExecutionRangeAccumulator();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this needed for the client side query metrics?

Copy link
Contributor Author

@j82w j82w Oct 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this data already in the client side request statistic? That already has the start and end time of each connection. Having this seemed like duplicate diagnostics. If there is a benefit please let me know and I'll add it back.

@kirankumarkolli kirankumarkolli merged commit 2725971 into master Oct 23, 2019
@kirankumarkolli kirankumarkolli deleted the users/jawilley/query/diagnostic_client branch October 23, 2019 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants