Implement an alternative to startup running twice when using WebApplicationFactory

[pdevito3]

So, I know it's by design, but I’m running into another instance where startup getting called twice when using WebApplicationFactory is causing me major headaches.

I added some code for this specific instance below, but the short version is that when I’m adding Auth into my API, it runs fine when doing startup normally, but when using the web host factory it's messing up my auth setup with a System.InvalidOperationException : Scheme already exists: Identity.Application error. error.

New Feature Request

Maybe I’m just not getting the best way to override things, but in my mind it makes more sense to have (at the the option of using) a distinct StartupTesting or something of that nature that can be run once to configure my testing host exactly how I want. This is how Laravel does it an it seems more manageable.
 
 Related to #19404
 

Details on this particular error

When using Auth, the API will run fine, but the integration tests will break, throwing a -------- System.InvalidOperationException : Scheme already exists: Identity.Application error.

I started googling for this and it seems like the main resolution is generally to remove AddDefaultIdentity to either stop a clash with IdentityHostingStartup or prevent IdentityHostintgStartup.cs from causing some overlap.

I'm not using AddDefaultIdentity and I'm not seeing a IdentityHostintgStartup.cs get generated, so I'm not quite sure what the deal is here. Presumably, something is calling AddAuthentication with the same identity scheme twice. This may be be due to CustomWebApplicationFactory running through startup multiple times, but I need to investigate more.

It does look like, when debugging any integration test that services.AddIdentity<ApplicationUser, IdentityRole>().AddEntityFrameworkStores<IdentityDbContext>().AddDefaultTokenProviders(); is getting hit twice and, when commenting that line out, I get a different error: -------- System.InvalidOperationException : Scheme already exists: Bearer which, again, is presumably happening because of startup getting run twice in CustomWebApplicationFactory.

WebAppFactory

namespace VetClinic.Api.Tests
{
    using Infrastructure.Persistence.Contexts;
    using Microsoft.AspNetCore.Hosting;
    using Microsoft.AspNetCore.Mvc.Testing;
    using Microsoft.EntityFrameworkCore;
    using Microsoft.Extensions.DependencyInjection;
    using Respawn;
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Net.Http;
    using System.Text;
    using System.Threading.Tasks;
    using WebApi;

    public class CustomWebApplicationFactory : WebApplicationFactory<Startup>
    {
        // checkpoint for respawn to clear the database when spenning up each time
        private static Checkpoint checkpoint = new Checkpoint
        {
            
        };

        protected override void ConfigureWebHost(IWebHostBuilder builder)
        {
            builder.UseEnvironment("Testing");

            builder.ConfigureServices(async services =>
            {
                services.AddEntityFrameworkInMemoryDatabase();

                // Create a new service provider.
                var provider = services
                    .AddEntityFrameworkInMemoryDatabase()
                    .BuildServiceProvider();

                // Add a database context (VetClinicDbContext) using an in-memory 
                // database for testing.
                services.AddDbContext<VetClinicDbContext>(options =>
                {
                    options.UseInMemoryDatabase("InMemoryDbForTesting");
                    options.UseInternalServiceProvider(provider);
                });

                // Build the service provider.
                var sp = services.BuildServiceProvider();

                // Create a scope to obtain a reference to the database
                // context (ApplicationDbContext).
                using (var scope = sp.CreateScope())
                {
                    var scopedServices = scope.ServiceProvider;
                    var db = scopedServices.GetRequiredService<VetClinicDbContext>();

                    // Ensure the database is created.
                    db.Database.EnsureCreated();

                    try
                    {
                        await checkpoint.Reset(db.Database.GetDbConnection());
                    }
                    catch
                    {
                    }
                }
            });
        }

        public HttpClient GetAnonymousClient()
        {
            return CreateClient();
        }
    }
}

Startup

namespace WebApi
{
    using Application;
    using Microsoft.AspNetCore.Builder;
    using Microsoft.AspNetCore.Hosting;
    using Microsoft.Extensions.Configuration;
    using Microsoft.Extensions.DependencyInjection;
    using Infrastructure.Persistence;
    using Infrastructure.Shared;
    using Infrastructure.Persistence.Seeders;
    using Infrastructure.Persistence.Contexts;
    using WebApi.Extensions;
    using Infrastructure.Identity;
    using Infrastructure.Identity.Entities;
    using Microsoft.AspNetCore.Identity;
    using Infrastructure.Identity.Seeders;
    using WebApi.Services;
    using Application.Interfaces;

    public class StartupDevelopment
    {
        public IConfiguration _config { get; }
        public StartupDevelopment(IConfiguration configuration)
        {
            _config = configuration;
        }

        // This method gets called by the runtime. Use this method to add services to the container.
        // For more information on how to configure your application, visit https://go.microsoft.com/fwlink/?LinkID=398940
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddCorsService("MyCorsPolicy");
            services.AddApplicationLayer();
            services.AddIdentityInfrastructure(_config);
            services.AddPersistenceInfrastructure(_config);
            services.AddSharedInfrastructure(_config);
            services.AddControllers()
                .AddNewtonsoftJson();
            services.AddApiVersioningExtension();
            services.AddHealthChecks();
            services.AddSingleton<ICurrentUserService, CurrentUserService>();

            #region Dynamic Services
            services.AddSwaggerExtension();
            #endregion
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            app.UseDeveloperExceptionPage();

            #region Entity Context Region - Do Not Delete

                using (var context = app.ApplicationServices.GetService<VetClinicDbContext>())
                {
                    context.Database.EnsureCreated();

                    #region VetClinicDbContext Seeder Region - Do Not Delete
                    
                    PetSeeder.SeedSamplePetData(app.ApplicationServices.GetService<VetClinicDbContext>());
                    VetSeeder.SeedSampleVetData(app.ApplicationServices.GetService<VetClinicDbContext>());
                    CitySeeder.SeedSampleCityData(app.ApplicationServices.GetService<VetClinicDbContext>());
                    #endregion
                }

            #endregion

            #region Identity Context Region - Do Not Delete

            var userManager = app.ApplicationServices.GetService<UserManager<ApplicationUser>>();
            var roleManager = app.ApplicationServices.GetService<RoleManager<IdentityRole>>();
            RoleSeeder.SeedDemoRolesAsync(roleManager);

            // user seeders -- do not delete this comment
            pdevitoSeeder.SeedUserAsync(userManager);

            #endregion

            app.UseCors("MyCorsPolicy");

            app.UseRouting();
            app.UseAuthentication();
            app.UseAuthorization();
            app.UseErrorHandlingMiddleware();
            app.UseEndpoints(endpoints =>
            {
                endpoints.MapHealthChecks("/api/health");
                endpoints.MapControllers();
            });

            #region Dynamic App
            app.UseSwaggerExtension();
            #endregion
        }
    }
}

Identity Extension

namespace Infrastructure.Identity
{
    using Application.Exceptions;
    using Application.Interfaces;
    using Application.Wrappers;
    using Domain.Settings;
    using Infrastructure.Identity.Entities;
    using Infrastructure.Identity.Services;
    using Microsoft.AspNetCore.Authentication.JwtBearer;
    using Microsoft.AspNetCore.Http;
    using Microsoft.AspNetCore.Identity;
    using Microsoft.EntityFrameworkCore;
    using Microsoft.Extensions.Configuration;
    using Microsoft.Extensions.DependencyInjection;
    using Microsoft.IdentityModel.Tokens;
    using Newtonsoft.Json;
    using System;
    using System.Text;

    public static class ServiceExtensions
    {
        public static void AddIdentityInfrastructure(this IServiceCollection services, IConfiguration configuration)
        {
            /*services.AddDbContext<IdentityDbContext>(options =>
                options.UseInMemoryDatabase("IdentityDb"));*/
            if (configuration.GetValue<bool>("UseInMemoryDatabase"))
            {
                services.AddDbContext<IdentityDbContext>(options =>
                    options.UseInMemoryDatabase("IdentityDb"));
            }
            else
            {
                services.AddDbContext<IdentityDbContext>(options =>
                options.UseSqlServer(
                    configuration.GetConnectionString("IdentityConnection"),
                    b => b.MigrationsAssembly(typeof(IdentityDbContext).Assembly.FullName)));
            }
            services.AddIdentity<ApplicationUser, IdentityRole>().AddEntityFrameworkStores<IdentityDbContext>().AddDefaultTokenProviders();

            #region Services
            services.AddScoped<IAccountService, AccountService>();
            #endregion

            // for craftsman updates to work appropriately, do not remove identity option lines
            services.Configure<IdentityOptions>(options =>
            {
                options.User.RequireUniqueEmail = true;

                options.Password.RequiredLength = 6;
                options.Password.RequireDigit = true;
                options.Password.RequireLowercase = true;
                options.Password.RequireUppercase = true;
                options.Password.RequireNonAlphanumeric = true;
            });

            services.Configure<JwtSettings>(configuration.GetSection("JwtSettings"));
            services.AddAuthentication(options =>
            {
                options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme;
                options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme;
            })
                .AddJwtBearer(o =>
                {
                    o.RequireHttpsMetadata = false;
                    o.SaveToken = false;
                    o.TokenValidationParameters = new TokenValidationParameters
                    {
                        ValidateIssuerSigningKey = true,
                        ValidateIssuer = true,
                        ValidateAudience = true,
                        ValidateLifetime = true,
                        ClockSkew = TimeSpan.Zero,
                        ValidIssuer = configuration["JwtSettings:Issuer"],
                        ValidAudience = configuration["JwtSettings:Audience"],
                        IssuerSigningKey = new SymmetricSecurityKey(Encoding.UTF8.GetBytes(configuration["JwtSettings:Key"]))
                    };
                    o.Events = new JwtBearerEvents()
                    {
                        OnAuthenticationFailed = c =>
                        {
                            c.NoResult();
                            c.Response.StatusCode = 500;
                            c.Response.ContentType = "text/plain";
                            return c.Response.WriteAsync(c.Exception.ToString());
                        },
                        OnChallenge = context =>
                        {
                            context.HandleResponse();
                            context.Response.StatusCode = 401;
                            context.Response.ContentType = "application/json";
                            var result = JsonConvert.SerializeObject(new Response<string>("You are not Authorized"));
                            return context.Response.WriteAsync(result);
                        },
                        OnForbidden = context =>
                        {
                            context.Response.StatusCode = 403;
                            context.Response.ContentType = "application/json";
                            var result = JsonConvert.SerializeObject(new Response<string>("You are not authorized to access this resource"));
                            return context.Response.WriteAsync(result);
                        },
                    };
                });
        }
    }
}

[pdevito3]

Realized I forgot to tag you guys for continuity.

@Tratcher
@davidfowl
@anurse

[Tratcher]

I'm confused by your base assertion here that Startup is running twice. That's not what the comments in the linked issue say.

You have multiple ConfigureServices methods and one runs after the other, but no single ConfigureServices method is being run twice, correct?

[pdevito3]

@Tratcher yeah, let me try to clarify.

In this instance and the linked instance, my code runs fine when doing a normal startup, but when running integration tests, it would seem that the entire ConfigureServices method is running twice, i.e. if I put a breakpoint on, any line in the AddIdentityInfrastructure method, it will get hit twice when running WebApplicationFactory.

Maybe I'm missing something with the proper configuration and operation of setting up integration tests, but even if that's the case, it isn't very intuitive as is. My point for the feature request is that, it would be great if we could have an alternative setup to just call a separate startup (e.g. StartupTesting) and call it a day. Simple and easy. No need to override anything in another startup and potentially run things multiple times.

[Tratcher]

Ok, we'll have to investigate that, it shouldn't run twice.

On another note, never call BuildServiceProvider, it messes up the DI service lifetimes. That DB initialization needs to happen later after the host/container get built.

[pdevito3]

Thanks, here's an example repo if you want to check out the bug in action.

Good tip, I'll be sure to update the BuildServiceProvider code. For what it's worth, the factory is based on eShopWeb, so a lot of people are probably getting bad info here. Maybe another good example that setting up the factory can be hard to wrap your head around for much of the community and having an option to have it work from a standard startup would be beneficial.

[Tratcher]

Thanks for the warning, I've filed dotnet-architecture/eShopOnWeb#465 to clean up the sample.

[pdevito3]

Thanks, will be sure to update this if I come across any other instances.

[ghost]

This issue has been resolved and has not had any activity for 1 day. It will be closed for housekeeping purposes.

See our Issue Management Policies for more information.

[Tratcher]

@mkArtakMSFT this isn't resolved. #26487 (comment) still needs investigation.

[ghost]

Thanks for contacting us.
We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. We will evaluate the request when we are planning the work for the next milestone. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[ghost]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

[RehanSaeed]

All I want from WebApplicationFactory is:

1. Add some services before Startup.ConfigureServices.
2. Startup.ConfigureServices is called only once.

I upgraded to .NET 5 and am now also seeing Startup.ConfigureServices called twice. I was using a TestStartup class inheriting from Startup to be able to override the ConfigureServices method and achieve Number 1 above which was working nicely. Changes to the WebApplicationFactory API has broken my code in every upgrade of .NET unfortunately. I don't really care how but is it possible to achieve the above two requirements?

[VictorioBerra]

I am having the exact same problem as @RehanSaeed I would expect if you override ConfigureWebHost in WebApplicationFactory and set UseStartup to a TestStartup class that the SUT Startup would not have ConfigureServices called as normal.

[ani-im]

Any updates on this?

[Belgian-Coder]

It seems like the issue is still there, are there known workarounds?

[j2jensen]

I doubt this is the reason for the problem mentioned in the original post, but in case anybody stumbles on this seeing the behavior I saw: I saw the Program code in a .NET 6 app getting executed twice because I had a WebApplicationFactory whose code references this.Server (as in clientBuilder.ConfigurePrimaryHttpMessageHandler(() => this.Server.CreateHandler())), but my tests applied further customizations via WithWebHostBuilder(), which provides a different instance of WebApplicationFactory. The call to this.Server caused the wrapped factory instance to spin up its own host. Changing that to clientBuilder.ConfigurePrimaryHttpMessageHandler(services => ((TestServer)services.GetRequiredService<IServer>()).CreateHandler()) fixed that problem.