Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[tests] Mismatched .bicep generated in AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode #5113

Closed
radical opened this issue Jul 29, 2024 · 5 comments · Fixed by #5187
Closed

[tests] Mismatched .bicep generated in AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode #5113

radical opened this issue Jul 29, 2024 · 5 comments · Fixed by #5187
Assignees
@radical
Labels
area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication blocking-clean-ci Blocking a green CI

Comments

@radical
Copy link
Member

radical commented Jul 29, 2024 •
edited by build-analysis bot
Loading

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=759660
Build error leg or test failing: Aspire.Hosting.Azure.Tests..AsAzurePostgresFlexibleServerViaRunMode
Pull request: #5099

  Failed Aspire.Hosting.Azure.Tests.AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode [30 ms]
  Error Message:
   Assert.Equal() Failure
                                 ↓ (pos 1390)
Expected: ···resource postgreSqlFirewallRule_T9qS4dcOa 'Microsoft.DBforPos···
Actual:   ···resource postgreSqlFlexibleServerDatabase_d6dHHd3yx 'Microsof···
                                 ↑ (pos 1390)
  Stack Trace:
     at Aspire.Hosting.Azure.Tests.AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode() in /_/tests/Aspire.Hosting.Azure.Tests/AzureBicepResourceTests.cs:line 1433
--- End of stack trace from previous location ---

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "Expected: .*resource postgreSqlFirewallRule",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=759660
Error message validated: [Expected: .*resource postgreSqlFirewallRule]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/29/2024 11:21:03 PM UTC

Report

Build Definition Test Pull Request
764179 dotnet/aspire Aspire.Hosting.Azure.Tests.AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode #5155
763604 dotnet/aspire Aspire.Hosting.Azure.Tests.AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode #5145
763593 dotnet/aspire Aspire.Hosting.Azure.Tests.AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode #4943
762906 dotnet/aspire Aspire.Hosting.Azure.Tests.AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode #5017
761326 dotnet/aspire Aspire.Hosting.Azure.Tests.AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode #5110
759974 dotnet/aspire Aspire.Hosting.Azure.Tests.AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode #5099
759660 dotnet/aspire Aspire.Hosting.Azure.Tests.AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode #5099

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 6 7
@radical radical added the blocking-clean-ci Blocking a green CI label Jul 29, 2024
@dotnet-issue-labeler dotnet-issue-labeler bot added the area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication label Jul 29, 2024
@radical
Copy link
Member Author

radical commented Jul 29, 2024

From the test log file:

[xUnit.net 00:00:06.20]     Aspire.Hosting.Azure.Tests.AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode [FAIL]
[xUnit.net 00:00:06.20]       Assert.Equal() Failure
[xUnit.net 00:00:06.20]                                        ↓ (pos 1390)
[xUnit.net 00:00:06.20]       Expected: ···resource postgreSqlFirewallRule_T9qS4dcOa 'Microsoft.DBforPos···
[xUnit.net 00:00:06.20]       Actual:   ···resource postgreSqlFlexibleServerDatabase_d6dHHd3yx 'Microsof···
[xUnit.net 00:00:06.21]                                        ↑ (pos 1390)
[xUnit.net 00:00:06.21]       Stack Trace:
[xUnit.net 00:00:06.21]         /_/tests/Aspire.Hosting.Azure.Tests/AzureBicepResourceTests.cs(1433,0): at Aspire.Hosting.Azure.Tests.AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode()
[xUnit.net 00:00:06.21]         --- End of stack trace from previous location ---
[xUnit.net 00:00:06.21]       Output:
[xUnit.net 00:00:06.21]         targetScope = 'resourceGroup'
[xUnit.net 00:00:06.21]         
[xUnit.net 00:00:06.21]         @description('')
[xUnit.net 00:00:06.21]         param location string = resourceGroup().location
[xUnit.net 00:00:06.21]         
[xUnit.net 00:00:06.21]         @description('')
[xUnit.net 00:00:06.21]         param administratorLogin string
[xUnit.net 00:00:06.21]         
[xUnit.net 00:00:06.21]         @secure()
[xUnit.net 00:00:06.21]         @description('')
[xUnit.net 00:00:06.21]         param administratorLoginPassword string
[xUnit.net 00:00:06.21]         
[xUnit.net 00:00:06.21]         @description('')
[xUnit.net 00:00:06.21]         param keyVaultName string
[xUnit.net 00:00:06.21]         
[xUnit.net 00:00:06.21]         
[xUnit.net 00:00:06.21]         resource keyVault_IeF8jZvXV 'Microsoft.KeyVault/vaults@2022-07-01' existing = {
[xUnit.net 00:00:06.21]           name: keyVaultName
[xUnit.net 00:00:06.21]         }
[xUnit.net 00:00:06.21]         
[xUnit.net 00:00:06.21]         resource postgreSqlFlexibleServer_hFZg1J8nf 'Microsoft.DBforPostgreSQL/flexibleServers@2023-03-01-preview' = {
[xUnit.net 00:00:06.21]           name: toLower(take('postgres${uniqueString(resourceGroup().id)}', 24))
[xUnit.net 00:00:06.21]           location: location
[xUnit.net 00:00:06.21]           tags: {
[xUnit.net 00:00:06.21]             'aspire-resource-name': 'postgres'
[xUnit.net 00:00:06.21]           }
[xUnit.net 00:00:06.21]           sku: {
[xUnit.net 00:00:06.21]             name: 'Standard_B1ms'
[xUnit.net 00:00:06.21]             tier: 'Burstable'
[xUnit.net 00:00:06.21]           }
[xUnit.net 00:00:06.21]           properties: {
[xUnit.net 00:00:06.21]             administratorLogin: administratorLogin
[xUnit.net 00:00:06.21]             administratorLoginPassword: administratorLoginPassword
[xUnit.net 00:00:06.21]             version: '16'
[xUnit.net 00:00:06.21]             storage: {
[xUnit.net 00:00:06.21]               storageSizeGB: 32
[xUnit.net 00:00:06.21]             }
[xUnit.net 00:00:06.21]             backup: {
[xUnit.net 00:00:06.21]               backupRetentionDays: 7
[xUnit.net 00:00:06.21]               geoRedundantBackup: 'Disabled'
[xUnit.net 00:00:06.21]             }
[xUnit.net 00:00:06.21]             highAvailability: {
[xUnit.net 00:00:06.21]               mode: 'Disabled'
[xUnit.net 00:00:06.21]             }
[xUnit.net 00:00:06.21]             availabilityZone: '1'
[xUnit.net 00:00:06.21]           }
[xUnit.net 00:00:06.21]         }
[xUnit.net 00:00:06.21]         
[xUnit.net 00:00:06.21]         resource postgreSqlFirewallRule_t5EgXW1q4 'Microsoft.DBforPostgreSQL/flexibleServers/firewallRules@2023-03-01-preview' = {
[xUnit.net 00:00:06.21]           parent: postgreSqlFlexibleServer_hFZg1J8nf
[xUnit.net 00:00:06.21]           name: 'AllowAllAzureIps'
[xUnit.net 00:00:06.21]           properties: {
[xUnit.net 00:00:06.21]             startIpAddress: '0.0.0.0'
[xUnit.net 00:00:06.21]             endIpAddress: '0.0.0.0'
[xUnit.net 00:00:06.21]           }
[xUnit.net 00:00:06.21]         }
[xUnit.net 00:00:06.21]         
[xUnit.net 00:00:06.21]         resource postgreSqlFlexibleServerDatabase_d6dHHd3yx 'Microsoft.DBforPostgreSQL/flexibleServers/databases@2023-03-01-preview' = {
[xUnit.net 00:00:06.21]           parent: postgreSqlFlexibleServer_hFZg1J8nf
[xUnit.net 00:00:06.21]           name: 'db'
[xUnit.net 00:00:06.21]           properties: {
[xUnit.net 00:00:06.21]           }
[xUnit.net 00:00:06.21]         }
[xUnit.net 00:00:06.21]         
[xUnit.net 00:00:06.21]         resource keyVaultSecret_Ddsc3HjrA 'Microsoft.KeyVault/vaults/secrets@2022-07-01' = {
[xUnit.net 00:00:06.21]           parent: keyVault_IeF8jZvXV
[xUnit.net 00:00:06.21]           name: 'connectionString'
[xUnit.net 00:00:06.21]           location: location
[xUnit.net 00:00:06.21]           properties: {
[xUnit.net 00:00:06.21]             value: 'Host=${postgreSqlFlexibleServer_hFZg1J8nf.properties.fullyQualifiedDomainName};Username=${administratorLogin};Password=${administratorLoginPassword}'
[xUnit.net 00:00:06.21]           }
[xUnit.net 00:00:06.21]         }
[xUnit.net 00:00:06.21]

Looking at the expected bicep in

aspire/tests/Aspire.Hosting.Azure.Tests/AzureBicepResourceTests.cs

Lines 1397 to 1431 in c3b47ef

resource postgreSqlFirewallRule_t5EgXW1q4 'Microsoft.DBforPostgreSQL/flexibleServers/firewallRules@2023-03-01-preview' = {
parent: postgreSqlFlexibleServer_hFZg1J8nf
name: 'AllowAllAzureIps'
properties: {
startIpAddress: '0.0.0.0'
endIpAddress: '0.0.0.0'
}
}
resource postgreSqlFirewallRule_T9qS4dcOa 'Microsoft.DBforPostgreSQL/flexibleServers/firewallRules@2023-03-01-preview' = {
parent: postgreSqlFlexibleServer_hFZg1J8nf
name: 'AllowAllIps'
properties: {
startIpAddress: '0.0.0.0'
endIpAddress: '255.255.255.255'
}
}
resource postgreSqlFlexibleServerDatabase_QJSbpnLQ9 'Microsoft.DBforPostgreSQL/flexibleServers/databases@2023-03-01-preview' = {
parent: postgreSqlFlexibleServer_hFZg1J8nf
name: 'dbName'
properties: {
}
}
resource keyVaultSecret_Ddsc3HjrA 'Microsoft.KeyVault/vaults/secrets@2022-07-01' = {
parent: keyVault_IeF8jZvXV
name: 'connectionString'
location: location
properties: {
value: 'Host=${postgreSqlFlexibleServer_hFZg1J8nf.properties.fullyQualifiedDomainName};Username=${administratorLogin};Password=${administratorLoginPassword}'
}
}
""";

.. the expected has:

resource postgreSqlFirewallRule_T9qS4dcOa 'Microsoft.DBforPostgreSQL/flexibleServers/firewallRules@2023-03-01-preview' = {
  parent: postgreSqlFlexibleServer_hFZg1J8nf
  name: 'AllowAllIps'
  properties: {
    startIpAddress: '0.0.0.0'
    endIpAddress: '255.255.255.255'
  }
}

resource postgreSqlFlexibleServerDatabase_QJSbpnLQ9 'Microsoft.DBforPostgreSQL/flexibleServers/databases@2023-03-01-preview' = {
  parent: postgreSqlFlexibleServer_hFZg1J8nf
  name: 'dbName'
  properties: {
  }
}

.. but the generated manifest has this instead:

resource postgreSqlFlexibleServerDatabase_d6dHHd3yx 'Microsoft.DBforPostgreSQL/flexibleServers/databases@2023-03-01-preview' = {
  parent: postgreSqlFlexibleServer_hFZg1J8nf
  name: 'db'
  properties: {
  }
}

The following code suggests that AllowAllIps is added for Run mode, but it is not being added here?

aspire/src/Aspire.Hosting.Azure.PostgreSQL/AzurePostgresExtensions.cs

Lines 73 to 78 in c3b47ef

if (builder.ApplicationBuilder.ExecutionContext.IsRunMode)
{
// Opens access to the Internet.
var openFirewallRule = new PostgreSqlFirewallRule(construct, "0.0.0.0", "255.255.255.255", postgres, "AllowAllIps");
}

cc @mitchdenny

@mitchdenny
Copy link
Member

@radical this build failure came from a PR, how do we know it wasn't something in the PR that caused it to fail?

(the error is surprising actually)

@radical
Copy link
Member Author

radical commented Jul 30, 2024

@radical this build failure came from a PR, how do we know it wasn't something in the PR that caused it to fail?

(the error is surprising actually)

Yeah, this was hit in #5099 which doesn't seem to have any related changes, AFAICS.

@radical
Copy link
Member Author

radical commented Aug 1, 2024

@mitchdenny this has been hit on 3 different PRs now.

@mitchdenny
Copy link
Member

Yep. I'm going to disable the test and try to see if I can figure out what is going on.

@mitchdenny mitchdenny mentioned this issue Aug 2, 2024
Closed
@radical radical self-assigned this Aug 5, 2024
radical added a commit to radical/aspire that referenced this issue Aug 5, 2024
@radical
[tests] ManifestUtils.GetManifestWithBicep: generate in a temporary path
f92bd77 
Avoid interfering with other tests by generating the bicep files in
per-test temporyary paths.

Issue: dotnet#5113
radical added a commit to radical/aspire that referenced this issue Aug 5, 2024
@radical
[tests] ManifestUtils.GetManifestWithBicep: generate in a temporary path
4d13422 
Avoid interfering with other tests by generating the bicep files in
per-test temporyary paths.

`AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode`
uses `postgres.module.bicep` also, and that file can get overwritten by
other tests like in `AzureBicepResourceTests`.

Issue: dotnet#5113
@radical radical mentioned this issue Aug 5, 2024
Merged
eerhardt pushed a commit that referenced this issue Aug 6, 2024
@radical
[tests] Fix some flaky tests in Aspire.Hosting.Azure.Tests (#5187)
cd1a61e 
* [tests] ValidateApplicationSamples: use a temporary path for generating the manifest

Other tests also generate and use `postgres.module.bicep` in the tests'
bindir, and these can run in parallel causing them to interfere with
each other.

Issue: #5174

* [tests] ManifestUtils.GetManifestWithBicep: generate in a temporary path

Avoid interfering with other tests by generating the bicep files in
per-test temporyary paths.

`AzureBicepResourceTests.AsAzurePostgresFlexibleServerViaRunMode`
uses `postgres.module.bicep` also, and that file can get overwritten by
other tests like in `AzureBicepResourceTests`.

Issue: #5113
@github-actions github-actions bot locked and limited conversation to collaborators Sep 5, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@radical radical

Labels
area-app-model Issues pertaining to the APIs in Aspire.Hosting, e.g. DistributedApplication blocking-clean-ci Blocking a green CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[tests] Fix some flaky tests in Aspire.Hosting.Azure.Tests radical/aspire
2 participants
@radical @mitchdenny