Flaky test: Aspire.Playground.Tests.AppHostTests.TestEndpointsReturnOk(testEndpoints: CosmosEndToEnd.AppHost (1 resources))

[davidfowl]

This PR is disabling the cosmos playground test and the cosmos component test because they fail repeatedly.

My guess is that the cosmos emulator is really slow to start and our timeouts are off. On top of that it does seem to randomly fail sometimes. I think we should change our testing to use the health checks implemented as part of WaitFor.

Trx:

Aspire.Playground.Tests(cosmos).zip

Test output

| [2024-09-20T23:05:02] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource cosmos/cosmos changed state: Starting
| [2024-09-20T23:05:02] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource cosmos/cosmos changed state: Starting -> Running
| [2024-09-20T23:05:02] Aspire.Hosting.DistributedApplication Information: Aspire version: 9.0.0-ci
| [2024-09-20T23:05:02] Aspire.Hosting.DistributedApplication Information: Distributed application starting.
| [2024-09-20T23:05:02] Aspire.Hosting.DistributedApplication Information: Application host directory is: /datadisks/disk1/work/A1600966/w/B1EC094F/e/playground/CosmosEndToEnd/CosmosEndToEnd.AppHost
| [2024-09-20T23:05:06] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource cosmos/cosmos changed state: Running -> Starting
| [2024-09-20T23:05:07] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource cosmos/cosmos changed state: Starting -> Hidden
| [2024-09-20T23:05:07] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource api/api changed state: Starting
| [2024-09-20T23:05:07] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource cosmos/cosmos-dztyymcd-60a5c8f1 changed state: Starting
| [2024-09-20T23:05:07] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource api/api changed state: Starting -> Waiting
| [2024-09-20T23:06:32] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource cosmos/cosmos-dztyymcd-60a5c8f1 changed state: Starting -> Running
| [2024-09-20T23:08:05] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource api/api changed state: Waiting -> Hidden
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.api Information: 1: 2024-09-20T23:05:07.1052337Z Waiting for resource 'cosmos' to enter the 'Running' state.
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.api Information: 2: 2024-09-20T23:06:32.3433361Z Waiting for resource 'cosmos' to become healthy.
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 1: 2024-09-20T23:05:02.9973266Z Skipping cosmos because it is not configured to be provisioned.
| [2024-09-20T23:08:05] Aspire.Hosting.DistributedApplication Information: Distributed application started. Press Ctrl+C to shut down.
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 1: 2024-09-20T23:06:28.8380000Z 3c379d216d9485882d8f16b02f4d804526e594962f4b023e2c7d3fa7c2089ee9
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 2: 2024-09-20T23:06:32.3380000Z 
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 3: 2024-09-20T23:06:32.3380000Z 3c379d216d9485882d8f16b02f4d804526e594962f4b023e2c7d3fa7c2089ee9
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 4: 2024-09-20T23:05:07.1440000Z Unable to find image 'mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest' locally
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 5: 2024-09-20T23:05:07.3870000Z latest: Pulling from cosmosdb/linux/azure-cosmos-emulator
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 6: 2024-09-20T23:05:07.3870000Z 7478e0ac0f23: Pulling fs layer
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 7: 2024-09-20T23:05:07.3870000Z 0188a21c6638: Pulling fs layer
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 8: 2024-09-20T23:05:07.3870000Z 90ef6e123cfc: Pulling fs layer
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 9: 2024-09-20T23:05:07.3870000Z 6dd3c0a85b11: Pulling fs layer
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 10: 2024-09-20T23:05:07.3870000Z f617387511f8: Pulling fs layer
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 11: 2024-09-20T23:05:07.3870000Z fcd490ccff95: Pulling fs layer
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 12: 2024-09-20T23:05:07.3870000Z 05a9392de48a: Pulling fs layer
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 13: 2024-09-20T23:05:07.3870000Z 0a94815a4f51: Pulling fs layer
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 14: 2024-09-20T23:05:07.3870000Z 8854cba1f530: Pulling fs layer
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 15: 2024-09-20T23:05:07.3870000Z 4f4fb700ef54: Pulling fs layer
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 16: 2024-09-20T23:05:07.3870000Z fcd490ccff95: Waiting
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 17: 2024-09-20T23:05:07.3870000Z 05a9392de48a: Waiting
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 18: 2024-09-20T23:05:07.3870000Z 0a94815a4f51: Waiting
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 19: 2024-09-20T23:05:07.3870000Z 8854cba1f530: Waiting
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 20: 2024-09-20T23:05:07.3870000Z 4f4fb700ef54: Waiting
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 21: 2024-09-20T23:05:07.3870000Z 6dd3c0a85b11: Waiting
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 22: 2024-09-20T23:05:07.3870000Z f617387511f8: Waiting
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 23: 2024-09-20T23:05:07.5090000Z 90ef6e123cfc: Verifying Checksum
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 24: 2024-09-20T23:05:07.5090000Z 90ef6e123cfc: Download complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 25: 2024-09-20T23:05:07.7250000Z 6dd3c0a85b11: Verifying Checksum
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 26: 2024-09-20T23:05:07.7250000Z 6dd3c0a85b11: Download complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 27: 2024-09-20T23:05:07.9190000Z f617387511f8: Verifying Checksum
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 28: 2024-09-20T23:05:07.9190000Z f617387511f8: Download complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 29: 2024-09-20T23:05:08.1640000Z 7478e0ac0f23: Verifying Checksum
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 30: 2024-09-20T23:05:08.1720000Z 7478e0ac0f23: Download complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 31: 2024-09-20T23:05:08.1830000Z 0188a21c6638: Verifying Checksum
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 32: 2024-09-20T23:05:08.1830000Z 0188a21c6638: Download complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 33: 2024-09-20T23:05:08.3380000Z fcd490ccff95: Verifying Checksum
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 34: 2024-09-20T23:05:08.3380000Z fcd490ccff95: Download complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 35: 2024-09-20T23:05:08.5430000Z 8854cba1f530: Verifying Checksum
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 36: 2024-09-20T23:05:08.5430000Z 8854cba1f530: Download complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 37: 2024-09-20T23:05:08.6910000Z 4f4fb700ef54: Verifying Checksum
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 38: 2024-09-20T23:05:08.6910000Z 4f4fb700ef54: Download complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 39: 2024-09-20T23:05:08.7990000Z 0a94815a4f51: Verifying Checksum
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 40: 2024-09-20T23:05:08.7990000Z 0a94815a4f51: Download complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 41: 2024-09-20T23:05:12.8430000Z 7478e0ac0f23: Pull complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 42: 2024-09-20T23:05:20.1880000Z 05a9392de48a: Verifying Checksum
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 43: 2024-09-20T23:05:20.1880000Z 05a9392de48a: Download complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 44: 2024-09-20T23:05:22.8090000Z 0188a21c6638: Pull complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 45: 2024-09-20T23:05:23.2290000Z 90ef6e123cfc: Pull complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 46: 2024-09-20T23:05:23.4450000Z 6dd3c0a85b11: Pull complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 47: 2024-09-20T23:05:23.6990000Z f617387511f8: Pull complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 48: 2024-09-20T23:05:24.5790000Z fcd490ccff95: Pull complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 49: 2024-09-20T23:06:27.2740000Z 05a9392de48a: Pull complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 50: 2024-09-20T23:06:28.3060000Z 0a94815a4f51: Pull complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 51: 2024-09-20T23:06:28.4760000Z 8854cba1f530: Pull complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 52: 2024-09-20T23:06:28.6190000Z 4f4fb700ef54: Pull complete
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 53: 2024-09-20T23:06:28.6760000Z Digest: sha256:171833ee465358fbbfdb5dc9af523ece5ca747dbf524a5c434e0c6773eaef0e0
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 54: 2024-09-20T23:06:28.6920000Z Status: Downloaded newer image for mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest
| [2024-09-20T23:08:05] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource api/api-rhjpgjhm-60a5c8f1 changed state: Running
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 55: 2024-09-20T23:07:29.1688948Z This is an evaluation version.  There are [178] days left in the evaluation period.
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 56: 2024-09-20T23:07:34.5038554Z 2.14.20.0 (728f9251)
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 57: 2024-09-20T23:07:34.5038947Z Copyright (C) Microsoft Corporation. All rights reserved.
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 58: 2024-09-20T23:07:37.6738462Z Starting
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 59: 2024-09-20T23:08:01.0743008Z Started 1/11 partitions
| [2024-09-20T23:08:05] CosmosEndToEnd.AppHost.Resources.cosmos Information: 60: 2024-09-20T23:08:02.6766382Z Started 2/11 partitions
| [2024-09-20T23:08:06] CosmosEndToEnd.AppHost.Resources.cosmos Information: 61: 2024-09-20T23:08:06.2135723Z Started 3/11 partitions
| [2024-09-20T23:08:07] CosmosEndToEnd.AppHost.Resources.api Information: 1: 2024-09-20T23:08:07.3630000Z \x1b[40m\x1b[32minfo\x1b[39m\x1b[22m\x1b[49m: Microsoft.Hosting.Lifetime[14]
| [2024-09-20T23:08:07] CosmosEndToEnd.AppHost.Resources.api Information: 2: 2024-09-20T23:08:07.3630000Z       Now listening on: http://localhost:36401
| [2024-09-20T23:08:07] CosmosEndToEnd.AppHost.Resources.api Information: 3: 2024-09-20T23:08:07.3690000Z \x1b[40m\x1b[32minfo\x1b[39m\x1b[22m\x1b[49m: Microsoft.Hosting.Lifetime[0]
| [2024-09-20T23:08:07] CosmosEndToEnd.AppHost.Resources.api Information: 4: 2024-09-20T23:08:07.3690000Z       Application started. Press Ctrl+C to shut down.
| [2024-09-20T23:08:07] CosmosEndToEnd.AppHost.Resources.api Information: 5: 2024-09-20T23:08:07.3710000Z \x1b[40m\x1b[32minfo\x1b[39m\x1b[22m\x1b[49m: Microsoft.Hosting.Lifetime[0]
| [2024-09-20T23:08:07] CosmosEndToEnd.AppHost.Resources.api Information: 6: 2024-09-20T23:08:07.3710000Z       Hosting environment: Development
| [2024-09-20T23:08:07] CosmosEndToEnd.AppHost.Resources.api Information: 7: 2024-09-20T23:08:07.3710000Z \x1b[40m\x1b[32minfo\x1b[39m\x1b[22m\x1b[49m: Microsoft.Hosting.Lifetime[0]
| [2024-09-20T23:08:07] CosmosEndToEnd.AppHost.Resources.api Information: 8: 2024-09-20T23:08:07.3710000Z       Content root path: /datadisks/disk1/work/A1600966/w/B1EC094F/e/playground/CosmosEndToEnd/CosmosEndToEnd.ApiService
| [2024-09-20T23:08:09] CosmosEndToEnd.AppHost.Resources.cosmos Information: 62: 2024-09-20T23:08:09.5970583Z Started 4/11 partitions
| [2024-09-20T23:08:13] CosmosEndToEnd.AppHost.Resources.cosmos Information: 63: 2024-09-20T23:08:12.9545667Z Started 5/11 partitions
| [2024-09-20T23:08:16] CosmosEndToEnd.AppHost.Resources.cosmos Information: 64: 2024-09-20T23:08:16.3174378Z Started 6/11 partitions
| [2024-09-20T23:08:19] CosmosEndToEnd.AppHost.Resources.cosmos Information: 65: 2024-09-20T23:08:19.6547325Z Started 7/11 partitions
| [2024-09-20T23:08:23] CosmosEndToEnd.AppHost.Resources.cosmos Information: 66: 2024-09-20T23:08:23.0395655Z Started 8/11 partitions
| [2024-09-20T23:08:26] CosmosEndToEnd.AppHost.Resources.cosmos Information: 67: 2024-09-20T23:08:26.3283835Z Started 9/11 partitions
| [2024-09-20T23:08:29] CosmosEndToEnd.AppHost.Resources.cosmos Information: 68: 2024-09-20T23:08:29.6502978Z Started 10/11 partitions
| [2024-09-20T23:08:31] CosmosEndToEnd.AppHost.Resources.cosmos Information: 69: 2024-09-20T23:08:31.1987123Z Started 11/11 partitions
| [2024-09-20T23:08:31] CosmosEndToEnd.AppHost.Resources.cosmos Information: 70: 2024-09-20T23:08:31.1988098Z Started
Calling endpoint 'http://localhost:46003/alive for resource 'api' in app 'CosmosEndToEnd.AppHost'
Calling endpoint 'http://localhost:46003/health for resource 'api' in app 'CosmosEndToEnd.AppHost'
Calling endpoint 'http://localhost:46003/ for resource 'api' in app 'CosmosEndToEnd.AppHost'
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 9: 2024-09-20T23:09:33.8610000Z \x1b[41m\x1b[30mfail\x1b[39m\x1b[22m\x1b[49m: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 10: 2024-09-20T23:09:33.8610000Z       An unhandled exception has occurred while executing the request.
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 11: 2024-09-20T23:09:33.8610000Z       Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: NotFound (404); Substatus: 1013; ActivityId: a048b47b-5854-4968-bb32-310aa3d96fb0; Reason: (
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 12: 2024-09-20T23:09:33.8610000Z       code : NotFound
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 13: 2024-09-20T23:09:33.8610000Z       message : Collection is not yet available for read. Please retry in some time.
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 14: 2024-09-20T23:09:33.8610000Z       ActivityId: a048b47b-5854-4968-bb32-310aa3d96fb0, Microsoft.Azure.Documents.Common/2.14.0
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 15: 2024-09-20T23:09:33.8610000Z       );
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 16: 2024-09-20T23:09:33.8610000Z          at Microsoft.Azure.Cosmos.ResponseMessage.EnsureSuccessStatusCode()
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 17: 2024-09-20T23:09:33.8610000Z          at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.ProcessMessage[T](ResponseMessage responseMessage, Func`2 createResponse)
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 18: 2024-09-20T23:09:33.8610000Z          at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.CreateContainerResponse(Container container, ResponseMessage responseMessage)
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 19: 2024-09-20T23:09:33.8610000Z          at Microsoft.Azure.Cosmos.DatabaseCore.CreateContainerIfNotExistsAsync(ContainerProperties containerProperties, ThroughputProperties throughputProperties, RequestOptions requestOptions, ITrace trace, CancellationToken cancellationToken)
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 20: 2024-09-20T23:09:33.8610000Z          at Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](String containerName, String databaseName, OperationType operationType, ITrace trace, Func`2 task, Func`2 openTelemetry, String operationName, RequestOptions requestOptions)
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 21: 2024-09-20T23:09:33.8610000Z          at Microsoft.Azure.Cosmos.ClientContextCore.OperationHelperWithRootTraceAsync[TResult](String operationName, String containerName, String databaseName, OperationType operationType, RequestOptions requestOptions, Func`2 task, Func`2 openTelemetry, TraceComponent traceComponent, TraceLevel traceLevel)
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 22: 2024-09-20T23:09:33.8610000Z          at Program.<>c.<<<Main>$>b__0_1>d.MoveNext() in /datadisks/disk1/work/A1600966/w/B1EC094F/e/playground/CosmosEndToEnd/CosmosEndToEnd.ApiService/Program.cs:line 23
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 23: 2024-09-20T23:09:33.8610000Z       --- End of stack trace from previous location ---
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 24: 2024-09-20T23:09:33.8610000Z          at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteTaskOfTFast>g__ExecuteAwaited|132_0[T](Task`1 task, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 25: 2024-09-20T23:09:33.8610000Z          at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)
| [2024-09-20T23:09:34] CosmosEndToEnd.AppHost.Resources.api Information: 26: 2024-09-20T23:09:33.8610000Z       --- Cosmos Diagnostics ---{"Summary":{"GatewayCalls":{"(404, 1013)":2,"(409, 0)":1}},"name":"CreateContainerIfNotExistsAsync","start datetime":"2024-09-20T23:09:33.451Z","duration in milliseconds":399.9713,"data":{"Client Configuration":{"Client Created Time Utc":"2024-09-20T23:08:32.0713911Z","MachineId":"vmId:bbf07a60-de3f-4974-bc0c-928f24cc346e","VM Region":"westus","NumberOfClientsCreated":1,"NumberOfActiveClients":1,"ConnectionMode":"Gateway","User Agent":"cosmos-netstandard-sdk/3.43.0|1|X64|Ubuntu 22.04.4 LTS|.NET 8.0.8|DN|Aspire","ConnectionConfig":{"gw":"(cps:50, urto:6, p:False, httpf: False)","rntbd":"(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: True, pr: ReuseUnicastPort)","other":"(ed:True, be:False)"},"ConsistencyConfig":"(consistency: NotSet, prgns:[], apprgn: )","ProcessorCount":2},"DistributedTraceId":"0993797f0c7e92889ed18bbb4c04a2e0"},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":236.4566,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":236.4148,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-20T23:08:42.3760722Z","cpu":20.869,"memory":1516752.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":98.9419,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:08:52.3766530Z","cpu":8.869,"memory":1514708.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0452,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:02.3770505Z","cpu":8.554,"memory":1512584.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0627,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:12.3773703Z","cpu":10.588,"memory":1506576.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0525,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:22.3777115Z","cpu":8.488,"memory":1513724.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0491,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:32.3780689Z","cpu":8.426,"memory":1504372.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0641,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":236.3942,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":236.3874,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":236.3365,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":236.3315,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":236.2501,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-20T23:09:33.4520972Z","DurationInMs":192.9931,"RequestUri":"https://127.0.0.1:38613/dbs/db/colls/entries","ResourceType":"Collection","HttpMethod":"GET","ActivityId":"a048b47b-5854-4968-bb32-310aa3d96fb0","StatusCode":"NotFound","ReasonPhrase":"Not Found"}]},"PointOperationStatisticsTraceDatum":{"Id":"PointOperationStatistics","ActivityId":"a048b47b-5854-4968-bb32-310aa3d96fb0","ResponseTimeUtc":"2024-09-20T23:09:33.6880515Z","StatusCode":404,"SubStatusCode":1013,"RequestCharge":0,"RequestUri":"dbs/db/colls/entries","ErrorMessage":null,"RequestSessionToken":null,"ResponseSessionToken":null,"BELatencyInMs":null}}}]}]}]}]}]}]},{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":69.7491,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":69.7131,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-20T23:08:42.3760722Z","cpu":20.869,"memory":1516752.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":98.9419,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:08:52.3766530Z","cpu":8.869,"memory":1514708.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0452,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:02.3770505Z","cpu":8.554,"memory":1512584.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0627,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:12.3773703Z","cpu":10.588,"memory":1506576.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0525,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:22.3777115Z","cpu":8.488,"memory":1513724.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0491,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:32.3780689Z","cpu":8.426,"memory":1504372.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0641,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":69.702,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":69.6924,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":69.6568,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":69.6529,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":69.561,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-20T23:09:33.6900908Z","DurationInMs":27.1843,"RequestUri":"https://127.0.0.1:38613/dbs/db/colls","ResourceType":"Collection","HttpMethod":"POST","ActivityId":"a048b47b-5854-4968-bb32-310aa3d96fb0","StatusCode":"Conflict","ReasonPhrase":"Conflict"}]},"PointOperationStatisticsTraceDatum":{"Id":"PointOperationStatistics","ActivityId":"a048b47b-5854-4968-bb32-310aa3d96fb0","ResponseTimeUtc":"2024-09-20T23:09:33.7593603Z","StatusCode":409,"SubStatusCode":0,"RequestCharge":0,"RequestUri":"dbs/db","ErrorMessage":null,"RequestSessionToken":null,"ResponseSessionToken":"0:-1#5","BELatencyInMs":"1.719"}}}]}]}]}]}]}]},{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":63.7738,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":63.7429,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-20T23:08:42.3760722Z","cpu":20.869,"memory":1516752.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":98.9419,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:08:52.3766530Z","cpu":8.869,"memory":1514708.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0452,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:02.3770505Z","cpu":8.554,"memory":1512584.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0627,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:12.3773703Z","cpu":10.588,"memory":1506576.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0525,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:22.3777115Z","cpu":8.488,"memory":1513724.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0491,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:32.3780689Z","cpu":8.426,"memory":1504372.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0641,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":63.7323,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":63.7261,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":63.6899,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":63.6865,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":63.612,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-20T23:09:33.7598488Z","DurationInMs":21.5893,"RequestUri":"https://127.0.0.1:38613/dbs/db/colls/entries","ResourceType":"Collection","HttpMethod":"GET","ActivityId":"a048b47b-5854-4968-bb32-310aa3d96fb0","StatusCode":"NotFound","ReasonPhrase":"Not Found"}]},"PointOperationStatisticsTraceDatum":{"Id":"PointOperationStatistics","ActivityId":"a048b47b-5854-4968-bb32-310aa3d96fb0","ResponseTimeUtc":"2024-09-20T23:09:33.8232251Z","StatusCode":404,"SubStatusCode":1013,"RequestCharge":0,"RequestUri":"dbs/db/colls/entries","ErrorMessage":null,"RequestSessionToken":null,"ResponseSessionToken":null,"BELatencyInMs":null}}}]}]}]}]}]}]}]}
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 27: 2024-09-20T23:09:35.3820000Z \x1b[41m\x1b[30mfail\x1b[39m\x1b[22m\x1b[49m: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 28: 2024-09-20T23:09:35.3820000Z       An unhandled exception has occurred while executing the request.
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 29: 2024-09-20T23:09:35.3820000Z       Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: NotFound (404); Substatus: 1013; ActivityId: 53f827e1-a60f-431a-a554-0d9b01429092; Reason: (
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 30: 2024-09-20T23:09:35.3820000Z       code : NotFound
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 31: 2024-09-20T23:09:35.3820000Z       message : Collection is not yet available for read. Please retry in some time.
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 32: 2024-09-20T23:09:35.3820000Z       ActivityId: 53f827e1-a60f-431a-a554-0d9b01429092, Microsoft.Azure.Documents.Common/2.14.0
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 33: 2024-09-20T23:09:35.3820000Z       );
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 34: 2024-09-20T23:09:35.3820000Z          at Microsoft.Azure.Cosmos.ResponseMessage.EnsureSuccessStatusCode()
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 35: 2024-09-20T23:09:35.3820000Z          at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.ProcessMessage[T](ResponseMessage responseMessage, Func`2 createResponse)
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 36: 2024-09-20T23:09:35.3820000Z          at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.CreateContainerResponse(Container container, ResponseMessage responseMessage)
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 37: 2024-09-20T23:09:35.3820000Z          at Microsoft.Azure.Cosmos.DatabaseCore.CreateContainerIfNotExistsAsync(ContainerProperties containerProperties, ThroughputProperties throughputProperties, RequestOptions requestOptions, ITrace trace, CancellationToken cancellationToken)
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 38: 2024-09-20T23:09:35.3820000Z          at Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](String containerName, String databaseName, OperationType operationType, ITrace trace, Func`2 task, Func`2 openTelemetry, String operationName, RequestOptions requestOptions)
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 39: 2024-09-20T23:09:35.3820000Z          at Microsoft.Azure.Cosmos.ClientContextCore.OperationHelperWithRootTraceAsync[TResult](String operationName, String containerName, String databaseName, OperationType operationType, RequestOptions requestOptions, Func`2 task, Func`2 openTelemetry, TraceComponent traceComponent, TraceLevel traceLevel)
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 40: 2024-09-20T23:09:35.3820000Z          at Program.<>c.<<<Main>$>b__0_1>d.MoveNext() in /datadisks/disk1/work/A1600966/w/B1EC094F/e/playground/CosmosEndToEnd/CosmosEndToEnd.ApiService/Program.cs:line 23
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 41: 2024-09-20T23:09:35.3820000Z       --- End of stack trace from previous location ---
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 42: 2024-09-20T23:09:35.3820000Z          at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteTaskOfTFast>g__ExecuteAwaited|132_0[T](Task`1 task, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 43: 2024-09-20T23:09:35.3820000Z          at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)
| [2024-09-20T23:09:35] CosmosEndToEnd.AppHost.Resources.api Information: 44: 2024-09-20T23:09:35.3820000Z       --- Cosmos Diagnostics ---{"Summary":{"GatewayCalls":{"(404, 1013)":2,"(409, 0)":1}},"name":"CreateContainerIfNotExistsAsync","start datetime":"2024-09-20T23:09:35.139Z","duration in milliseconds":242.1577,"data":{"Client Configuration":{"Client Created Time Utc":"2024-09-20T23:08:32.0713911Z","MachineId":"vmId:bbf07a60-de3f-4974-bc0c-928f24cc346e","VM Region":"westus","NumberOfClientsCreated":1,"NumberOfActiveClients":1,"ConnectionMode":"Gateway","User Agent":"cosmos-netstandard-sdk/3.43.0|1|X64|Ubuntu 22.04.4 LTS|.NET 8.0.8|DN|Aspire","ConnectionConfig":{"gw":"(cps:50, urto:6, p:False, httpf: False)","rntbd":"(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: True, pr: ReuseUnicastPort)","other":"(ed:True, be:False)"},"ConsistencyConfig":"(consistency: NotSet, prgns:[], apprgn: )","ProcessorCount":2},"DistributedTraceId":"5f46db6a5cf50c48664b97c5d804918f"},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":120.2838,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":120.2614,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-20T23:08:42.3760722Z","cpu":20.869,"memory":1516752.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":98.9419,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:08:52.3766530Z","cpu":8.869,"memory":1514708.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0452,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:02.3770505Z","cpu":8.554,"memory":1512584.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0627,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:12.3773703Z","cpu":10.588,"memory":1506576.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0525,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:22.3777115Z","cpu":8.488,"memory":1513724.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0491,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:32.3780689Z","cpu":8.426,"memory":1504372.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0641,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":120.2415,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":120.2332,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":120.2018,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":120.1956,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":120.121,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-20T23:09:35.1393929Z","DurationInMs":73.1081,"RequestUri":"https://127.0.0.1:38613/dbs/db/colls/entries","ResourceType":"Collection","HttpMethod":"GET","ActivityId":"53f827e1-a60f-431a-a554-0d9b01429092","StatusCode":"NotFound","ReasonPhrase":"Not Found"}]},"PointOperationStatisticsTraceDatum":{"Id":"PointOperationStatistics","ActivityId":"53f827e1-a60f-431a-a554-0d9b01429092","ResponseTimeUtc":"2024-09-20T23:09:35.2592180Z","StatusCode":404,"SubStatusCode":1013,"RequestCharge":0,"RequestUri":"dbs/db/colls/entries","ErrorMessage":null,"RequestSessionToken":null,"ResponseSessionToken":null,"BELatencyInMs":null}}}]}]}]}]}]}]},{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":55.8667,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":55.8302,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-20T23:08:42.3760722Z","cpu":20.869,"memory":1516752.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":98.9419,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:08:52.3766530Z","cpu":8.869,"memory":1514708.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0452,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:02.3770505Z","cpu":8.554,"memory":1512584.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0627,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:12.3773703Z","cpu":10.588,"memory":1506576.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0525,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:22.3777115Z","cpu":8.488,"memory":1513724.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0491,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:32.3780689Z","cpu":8.426,"memory":1504372.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0641,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":55.8187,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":55.8133,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":55.7824,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":55.7787,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":55.6869,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-20T23:09:35.2598278Z","DurationInMs":11.9817,"RequestUri":"https://127.0.0.1:38613/dbs/db/colls","ResourceType":"Collection","HttpMethod":"POST","ActivityId":"53f827e1-a60f-431a-a554-0d9b01429092","StatusCode":"Conflict","ReasonPhrase":"Conflict"}]},"PointOperationStatisticsTraceDatum":{"Id":"PointOperationStatistics","ActivityId":"53f827e1-a60f-431a-a554-0d9b01429092","ResponseTimeUtc":"2024-09-20T23:09:35.3151782Z","StatusCode":409,"SubStatusCode":0,"RequestCharge":0,"RequestUri":"dbs/db","ErrorMessage":null,"RequestSessionToken":null,"ResponseSessionToken":"0:-1#5","BELatencyInMs":"1.658"}}}]}]}]}]}]}]},{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":63.983,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":63.9613,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-20T23:08:42.3760722Z","cpu":20.869,"memory":1516752.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":98.9419,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:08:52.3766530Z","cpu":8.869,"memory":1514708.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0452,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:02.3770505Z","cpu":8.554,"memory":1512584.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0627,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:12.3773703Z","cpu":10.588,"memory":1506576.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0525,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:22.3777115Z","cpu":8.488,"memory":1513724.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0491,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:32.3780689Z","cpu":8.426,"memory":1504372.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0641,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":63.9482,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":63.9426,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":63.9001,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":63.8969,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":63.6299,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-20T23:09:35.3158743Z","DurationInMs":20.3154,"RequestUri":"https://127.0.0.1:38613/dbs/db/colls/entries","ResourceType":"Collection","HttpMethod":"GET","ActivityId":"53f827e1-a60f-431a-a554-0d9b01429092","StatusCode":"NotFound","ReasonPhrase":"Not Found"}]},"PointOperationStatisticsTraceDatum":{"Id":"PointOperationStatistics","ActivityId":"53f827e1-a60f-431a-a554-0d9b01429092","ResponseTimeUtc":"2024-09-20T23:09:35.3792696Z","StatusCode":404,"SubStatusCode":1013,"RequestCharge":0,"RequestUri":"dbs/db/colls/entries","ErrorMessage":null,"RequestSessionToken":null,"ResponseSessionToken":null,"BELatencyInMs":null}}}]}]}]}]}]}]}]}
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 45: 2024-09-20T23:09:37.8780000Z \x1b[41m\x1b[30mfail\x1b[39m\x1b[22m\x1b[49m: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 46: 2024-09-20T23:09:37.8780000Z       An unhandled exception has occurred while executing the request.
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 47: 2024-09-20T23:09:37.8780000Z       Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: RequestTimeout (408); Substatus: 0; ActivityId: e5857245-f6e2-4588-a036-2b3d2cd7912e; Reason: (GatewayStoreClient Request Timeout. Start Time UTC:09/20/2024 23:08:32; Total Duration:65004.1641 Ms; Request Timeout 65000 Ms; Http Client Timeout:65000 Ms; Activity id: e5857245-f6e2-4588-a036-2b3d2cd7912e;);
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 48: 2024-09-20T23:09:37.8780000Z        ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 49: 2024-09-20T23:09:37.8780000Z        ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 50: 2024-09-20T23:09:37.8780000Z        ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 51: 2024-09-20T23:09:37.8780000Z        ---> System.Net.Sockets.SocketException (125): Operation canceled
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 52: 2024-09-20T23:09:37.8780000Z          --- End of inner exception stack trace ---
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 53: 2024-09-20T23:09:37.8780000Z          at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 54: 2024-09-20T23:09:37.8780000Z          at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 55: 2024-09-20T23:09:37.8780000Z          at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 56: 2024-09-20T23:09:37.8780000Z          at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 57: 2024-09-20T23:09:37.8780000Z          at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 58: 2024-09-20T23:09:37.8780000Z          at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 59: 2024-09-20T23:09:37.8780000Z          at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 60: 2024-09-20T23:09:37.8780000Z          --- End of inner exception stack trace ---
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 61: 2024-09-20T23:09:37.8780000Z          at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 62: 2024-09-20T23:09:37.8780000Z          at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 63: 2024-09-20T23:09:37.8780000Z          at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 64: 2024-09-20T23:09:37.8780000Z          at System.Net.Http.Metrics.MetricsHandler.SendAsyncWithMetrics(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 65: 2024-09-20T23:09:37.8780000Z          at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 66: 2024-09-20T23:09:37.8780000Z          at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 67: 2024-09-20T23:09:37.8780000Z          --- End of inner exception stack trace ---
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 68: 2024-09-20T23:09:37.8780000Z          at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 69: 2024-09-20T23:09:37.8780000Z          at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 70: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.CosmosHttpClientCore.ExecuteHttpHelperAsync(HttpRequestMessage requestMessage, ResourceType resourceType, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 71: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.CosmosHttpClientCore.SendHttpHelperAsync(Func`1 createRequestMessageAsync, ResourceType resourceType, HttpTimeoutPolicy timeoutPolicy, IClientSideRequestStatistics clientSideRequestStatistics, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 72: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.GatewayStoreClient.InvokeAsync(DocumentServiceRequest request, ResourceType resourceType, Uri physicalAddress, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 73: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.GatewayStoreModel.ProcessMessageAsync(DocumentServiceRequest request, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 74: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.Handlers.TransportHandler.ProcessMessageAsync(RequestMessage request, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 75: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.Handlers.TransportHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 76: 2024-09-20T23:09:37.8780000Z          --- End of inner exception stack trace ---
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 77: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.ResponseMessage.EnsureSuccessStatusCode()
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 78: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.ProcessMessage[T](ResponseMessage responseMessage, Func`2 createResponse)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 79: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.CreateContainerResponse(Container container, ResponseMessage responseMessage)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 80: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.DatabaseCore.CreateContainerIfNotExistsAsync(ContainerProperties containerProperties, ThroughputProperties throughputProperties, RequestOptions requestOptions, ITrace trace, CancellationToken cancellationToken)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 81: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](String containerName, String databaseName, OperationType operationType, ITrace trace, Func`2 task, Func`2 openTelemetry, String operationName, RequestOptions requestOptions)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 82: 2024-09-20T23:09:37.8780000Z          at Microsoft.Azure.Cosmos.ClientContextCore.OperationHelperWithRootTraceAsync[TResult](String operationName, String containerName, String databaseName, OperationType operationType, RequestOptions requestOptions, Func`2 task, Func`2 openTelemetry, TraceComponent traceComponent, TraceLevel traceLevel)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 83: 2024-09-20T23:09:37.8780000Z          at Program.<>c.<<<Main>$>b__0_1>d.MoveNext() in /datadisks/disk1/work/A1600966/w/B1EC094F/e/playground/CosmosEndToEnd/CosmosEndToEnd.ApiService/Program.cs:line 23
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 84: 2024-09-20T23:09:37.8780000Z       --- End of stack trace from previous location ---
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 85: 2024-09-20T23:09:37.8780000Z          at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteTaskOfTFast>g__ExecuteAwaited|132_0[T](Task`1 task, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 86: 2024-09-20T23:09:37.8780000Z          at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)
| [2024-09-20T23:09:38] CosmosEndToEnd.AppHost.Resources.api Information: 87: 2024-09-20T23:09:37.8780000Z       --- Cosmos Diagnostics ---{"Summary":{"GatewayCalls":{"(404, 0)":1,"(0, 0)":1}},"name":"CreateContainerIfNotExistsAsync","start datetime":"2024-09-20T23:08:32.773Z","duration in milliseconds":65102.7358,"data":{"Client Configuration":{"Client Created Time Utc":"2024-09-20T23:08:32.0713911Z","MachineId":"vmId:bbf07a60-de3f-4974-bc0c-928f24cc346e","VM Region":"westus","NumberOfClientsCreated":1,"NumberOfActiveClients":1,"ConnectionMode":"Gateway","User Agent":"cosmos-netstandard-sdk/3.43.0|1|X64|Ubuntu 22.04.4 LTS|.NET 8.0.8|DN|Aspire","ConnectionConfig":{"gw":"(cps:50, urto:6, p:False, httpf: False)","rntbd":"(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: True, pr: ReuseUnicastPort)","other":"(ed:True, be:False)"},"ConsistencyConfig":"(consistency: NotSet, prgns:[], apprgn: )","ProcessorCount":2},"DistributedTraceId":"0b64a33571c09807bdf7130808a3d2a5"},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":66.8706,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":66.83,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-20T23:08:32.3665067Z","cpu":0.000,"memory":1535520.000,"threadInfo":{"isThreadStarving":"no info","availableThreads":32765,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":66.8187,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":66.8023,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":66.7286,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":66.7248,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":66.6238,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-20T23:08:32.7843822Z","DurationInMs":60.2122,"RequestUri":"https://127.0.0.1:38613/dbs/db/colls/entries","ResourceType":"Collection","HttpMethod":"GET","ActivityId":"e5857245-f6e2-4588-a036-2b3d2cd7912e","StatusCode":"NotFound","ReasonPhrase":"Not Found"}]},"PointOperationStatisticsTraceDatum":{"Id":"PointOperationStatistics","ActivityId":"e5857245-f6e2-4588-a036-2b3d2cd7912e","ResponseTimeUtc":"2024-09-20T23:08:32.8464358Z","StatusCode":404,"SubStatusCode":0,"RequestCharge":2,"RequestUri":"dbs/db/colls/entries","ErrorMessage":null,"RequestSessionToken":null,"ResponseSessionToken":"0:-1#4","BELatencyInMs":"0.323"}}}]}]}]}]}]}]},{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":65007.2805,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":65007.2153,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-20T23:08:42.3760722Z","cpu":20.869,"memory":1516752.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":98.9419,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:08:52.3766530Z","cpu":8.869,"memory":1514708.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0452,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:02.3770505Z","cpu":8.554,"memory":1512584.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0627,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:12.3773703Z","cpu":10.588,"memory":1506576.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0525,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:22.3777115Z","cpu":8.488,"memory":1513724.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0491,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:32.3780689Z","cpu":8.426,"memory":1504372.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0641,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":65007.2039,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":65007.1984,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":65007.0698,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":65007.0648,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":65006.5916,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-20T23:08:32.8664240Z","DurationInMs":65003.9484,"RequestUri":"https://127.0.0.1:38613/dbs/db/colls","ResourceType":"Collection","HttpMethod":"POST","ActivityId":"e5857245-f6e2-4588-a036-2b3d2cd7912e","ExceptionType":"System.Threading.Tasks.TaskCanceledException","ExceptionMessage":"The operation was canceled."}]}}}]}]}]}]}]}]}]}
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 88: 2024-09-20T23:09:39.2380000Z \x1b[41m\x1b[30mfail\x1b[39m\x1b[22m\x1b[49m: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 89: 2024-09-20T23:09:39.2380000Z       An unhandled exception has occurred while executing the request.
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 90: 2024-09-20T23:09:39.2380000Z       Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: NotFound (404); Substatus: 1013; ActivityId: a0c44d1e-3dde-4088-a776-3b1df0189307; Reason: (
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 91: 2024-09-20T23:09:39.2380000Z       code : NotFound
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 92: 2024-09-20T23:09:39.2380000Z       message : Collection is not yet available for read. Please retry in some time.
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 93: 2024-09-20T23:09:39.2380000Z       ActivityId: a0c44d1e-3dde-4088-a776-3b1df0189307, Microsoft.Azure.Documents.Common/2.14.0
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 94: 2024-09-20T23:09:39.2380000Z       );
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 95: 2024-09-20T23:09:39.2380000Z          at Microsoft.Azure.Cosmos.ResponseMessage.EnsureSuccessStatusCode()
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 96: 2024-09-20T23:09:39.2380000Z          at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.ProcessMessage[T](ResponseMessage responseMessage, Func`2 createResponse)
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 97: 2024-09-20T23:09:39.2380000Z          at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.CreateContainerResponse(Container container, ResponseMessage responseMessage)
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 98: 2024-09-20T23:09:39.2380000Z          at Microsoft.Azure.Cosmos.DatabaseCore.CreateContainerIfNotExistsAsync(ContainerProperties containerProperties, ThroughputProperties throughputProperties, RequestOptions requestOptions, ITrace trace, CancellationToken cancellationToken)
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 99: 2024-09-20T23:09:39.2380000Z          at Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](String containerName, String databaseName, OperationType operationType, ITrace trace, Func`2 task, Func`2 openTelemetry, String operationName, RequestOptions requestOptions)
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 100: 2024-09-20T23:09:39.2380000Z          at Microsoft.Azure.Cosmos.ClientContextCore.OperationHelperWithRootTraceAsync[TResult](String operationName, String containerName, String databaseName, OperationType operationType, RequestOptions requestOptions, Func`2 task, Func`2 openTelemetry, TraceComponent traceComponent, TraceLevel traceLevel)
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 101: 2024-09-20T23:09:39.2380000Z          at Program.<>c.<<<Main>$>b__0_1>d.MoveNext() in /datadisks/disk1/work/A1600966/w/B1EC094F/e/playground/CosmosEndToEnd/CosmosEndToEnd.ApiService/Program.cs:line 23
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 102: 2024-09-20T23:09:39.2380000Z       --- End of stack trace from previous location ---
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 103: 2024-09-20T23:09:39.2380000Z          at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteTaskOfTFast>g__ExecuteAwaited|132_0[T](Task`1 task, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 104: 2024-09-20T23:09:39.2380000Z          at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)
| [2024-09-20T23:09:39] CosmosEndToEnd.AppHost.Resources.api Information: 105: 2024-09-20T23:09:39.2380000Z       --- Cosmos Diagnostics ---{"Summary":{"GatewayCalls":{"(404, 1013)":2,"(409, 0)":1}},"name":"CreateContainerIfNotExistsAsync","start datetime":"2024-09-20T23:09:39.018Z","duration in milliseconds":219.4057,"data":{"Client Configuration":{"Client Created Time Utc":"2024-09-20T23:08:32.0713911Z","MachineId":"vmId:bbf07a60-de3f-4974-bc0c-928f24cc346e","VM Region":"westus","NumberOfClientsCreated":1,"NumberOfActiveClients":1,"ConnectionMode":"Gateway","User Agent":"cosmos-netstandard-sdk/3.43.0|1|X64|Ubuntu 22.04.4 LTS|.NET 8.0.8|DN|Aspire","ConnectionConfig":{"gw":"(cps:50, urto:6, p:False, httpf: False)","rntbd":"(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: True, pr: ReuseUnicastPort)","other":"(ed:True, be:False)"},"ConsistencyConfig":"(consistency: NotSet, prgns:[], apprgn: )","ProcessorCount":2},"DistributedTraceId":"4e42a85c045f6af4f9686df365e74606"},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":93.4754,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":93.4477,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-20T23:08:42.3760722Z","cpu":20.869,"memory":1516752.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":98.9419,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:08:52.3766530Z","cpu":8.869,"memory":1514708.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0452,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:02.3770505Z","cpu":8.554,"memory":1512584.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0627,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:12.3773703Z","cpu":10.588,"memory":1506576.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0525,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:22.3777115Z","cpu":8.488,"memory":1513724.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0491,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:32.3780689Z","cpu":8.426,"memory":1504372.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0641,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":93.4296,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":93.424,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":93.3936,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":93.389,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":93.318,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-20T23:09:39.0183483Z","DurationInMs":49.9562,"RequestUri":"https://127.0.0.1:38613/dbs/db/colls/entries","ResourceType":"Collection","HttpMethod":"GET","ActivityId":"a0c44d1e-3dde-4088-a776-3b1df0189307","StatusCode":"NotFound","ReasonPhrase":"Not Found"}]},"PointOperationStatisticsTraceDatum":{"Id":"PointOperationStatistics","ActivityId":"a0c44d1e-3dde-4088-a776-3b1df0189307","ResponseTimeUtc":"2024-09-20T23:09:39.1113546Z","StatusCode":404,"SubStatusCode":1013,"RequestCharge":0,"RequestUri":"dbs/db/colls/entries","ErrorMessage":null,"RequestSessionToken":null,"ResponseSessionToken":null,"BELatencyInMs":null}}}]}]}]}]}]}]},{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":55.657,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":55.6305,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-20T23:08:42.3760722Z","cpu":20.869,"memory":1516752.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":98.9419,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:08:52.3766530Z","cpu":8.869,"memory":1514708.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0452,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:02.3770505Z","cpu":8.554,"memory":1512584.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0627,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:12.3773703Z","cpu":10.588,"memory":1506576.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0525,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:22.3777115Z","cpu":8.488,"memory":1513724.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0491,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:32.3780689Z","cpu":8.426,"memory":1504372.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0641,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":55.6189,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":55.6131,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":55.5738,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":55.569,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":55.488,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-20T23:09:39.1119867Z","DurationInMs":14.6132,"RequestUri":"https://127.0.0.1:38613/dbs/db/colls","ResourceType":"Collection","HttpMethod":"POST","ActivityId":"a0c44d1e-3dde-4088-a776-3b1df0189307","StatusCode":"Conflict","ReasonPhrase":"Conflict"}]},"PointOperationStatisticsTraceDatum":{"Id":"PointOperationStatistics","ActivityId":"a0c44d1e-3dde-4088-a776-3b1df0189307","ResponseTimeUtc":"2024-09-20T23:09:39.1672306Z","StatusCode":409,"SubStatusCode":0,"RequestCharge":0,"RequestUri":"dbs/db","ErrorMessage":null,"RequestSessionToken":null,"ResponseSessionToken":"0:-1#5","BELatencyInMs":"2.144"}}}]}]}]}]}]}]},{"name":"Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler","duration in milliseconds":68.7727,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler","duration in milliseconds":68.7438,"data":{"System Info":{"systemHistory":[{"dateUtc":"2024-09-20T23:08:42.3760722Z","cpu":20.869,"memory":1516752.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":98.9419,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:08:52.3766530Z","cpu":8.869,"memory":1514708.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0452,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:02.3770505Z","cpu":8.554,"memory":1512584.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0627,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:12.3773703Z","cpu":10.588,"memory":1506576.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0525,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:22.3777115Z","cpu":8.488,"memory":1513724.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0491,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0},{"dateUtc":"2024-09-20T23:09:32.3780689Z","cpu":8.426,"memory":1504372.000,"threadInfo":{"isThreadStarving":"False","threadWaitIntervalInMs":0.0641,"availableThreads":32766,"minThreads":2,"maxThreads":32767},"numberOfOpenTcpConnection":0}]}},"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TelemetryHandler","duration in milliseconds":68.7332,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RetryHandler","duration in milliseconds":68.7256,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.RouterHandler","duration in milliseconds":68.6894,"children":[{"name":"Microsoft.Azure.Cosmos.Handlers.TransportHandler","duration in milliseconds":68.6836,"children":[{"name":"Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request","duration in milliseconds":68.5657,"data":{"Client Side Request Stats":{"Id":"AggregatedClientSideRequestStatistics","ContactedReplicas":[],"RegionsContacted":[],"FailedReplicas":[],"AddressResolutionStatistics":[],"StoreResponseStatistics":[],"HttpResponseStats":[{"StartTimeUTC":"2024-09-20T23:09:39.1677182Z","DurationInMs":23.9371,"RequestUri":"https://127.0.0.1:38613/dbs/db/colls/entries","ResourceType":"Collection","HttpMethod":"GET","ActivityId":"a0c44d1e-3dde-4088-a776-3b1df0189307","StatusCode":"NotFound","ReasonPhrase":"Not Found"}]},"PointOperationStatisticsTraceDatum":{"Id":"PointOperationStatistics","ActivityId":"a0c44d1e-3dde-4088-a776-3b1df0189307","ResponseTimeUtc":"2024-09-20T23:09:39.2360030Z","StatusCode":404,"SubStatusCode":1013,"RequestCharge":0,"RequestUri":"dbs/db/colls/entries","ErrorMessage":null,"RequestSessionToken":null,"ResponseSessionToken":null,"BELatencyInMs":null}}}]}]}]}]}]}]}]}
| [2024-09-20T23:10:31] Aspire.Hosting.ApplicationModel.ResourceNotificationService Debug: Resource cosmos/cosmos-dztyymcd-60a5c8f1 changed state: Running -> Stopping