AddContainer does not have support for docker network

[thijscornelis]

Hi,

Is there some roadmap for adding more docker container capabilities? Right now, I see that there is support for Volumes and Replicas using the ResourceAnnotations.

builder.AddContainer(..)
    .WithAnnotation(new VolumeMountAnnotation(..))
    .WithAnnotation(new ReplicaAnnotation(..))

I was looking to create a set of containers that use a docker network. If it's something that is not meant to be done using ResourceAnnotations, could you guide me on how we should do it? I see no option to add arguments to the 'docker run' command?

Greetings!

P.S. I'm loving this project already :)

[karolz-ms]

What is the benefit of using Docker network for your app?

[davidfowl]

We shouldn't add support for docker networks but we should make a default network so that containers can reference each other by host name.

[thijscornelis]

What is the benefit of using Docker network for your app?

I was following a tutorial to get an Elastic container up & running. Didn't think twice if I really needed a separate docker network, as I'm quite new to containers.

[davidfowl]

That makes sense @thijscornelis. Can you share the tutorial?

[thijscornelis]

That makes sense @thijscornelis. Can you share the tutorial?

Sure, here you go: https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html

[Blind-Striker]

Another potential approach could be to leverage Dotnet Test containers and integrate it to Aspire. Testcontainers is a widely-used and well-established library within the community, offering robust and tested functionalities.

If this aligns with the project goals, I'm ready to open an issue to explore integrating Testcontainers. Thoughts?

Dotnet Testcontainers GitHub

[karolz-ms]

I agree with @davidfowl, we should be using a dedicated Docker network for running an Aspire app (and also use the Aspire component names for container names, instead of Docker auto-generated names).

@Blind-Striker thank you for pointing out the Testcontainers project, looks interesting, I will take a closer look.

[luizcarlosfaria]

We shouldn't add support for docker networks but we should make a default network so that containers can reference each other by host name.

Default networks in docker work's some different.

We can see on Docker Docs "Containers that attach to a custom network use Docker's embedded DNS server. The embedded DNS server forwards external DNS lookups to the DNS servers configured on the host." at https://docs.docker.com/network/#dns-services

Service discovery is part of custom docker networks, and when we connect some containers at this network, we're expecting that some containers talk to each other.

Kafka and zookeper, etcd and other service tools, elk stack or anything else, like a kong, nginx envoy etc need a custom docker network to use a docker service discovery.

[3GDXC]

IMHO Network configuration would be good as it would allow true infrastructure as code and follow what best practices with regards layer separation and firewall/port configuration testing in the developer environments.

[davidfowl]

IMHO Network configuration would be good as it would allow true infrastructure as code and follow what best practices with regards layer separation and firewall/port configuration testing in the developer environments.

Can you expand on this? What does that mean exactly?

[3GDXC]

IMHO Network configuration would be good as it would allow true infrastructure as code and follow what best practices with regards layer separation and firewall/port configuration testing in the developer environments.

Can you expand on this? What does that mean exactly?

firstly, happy new year @davidfowl and team; hope you all had a good holiday period.
and apologies for the cryptic description above.

We have a number guiding principles from our security and infrastructure teams; that we are required to follow when designing application/system architecture. These principles are all around the topic of secure by design; and state we should separating application from it's services (not owned by the application its self) at a network layer especially common/shared services like caching/messaging/database. The separation may be provided by the network via vlan routing, firewall or reverse-proxy allowing traffic between groups of services by port mappings.

To support the secure by design mandated by our security & infrastructure teams have tooling that is used map network designs (visual) or reverse engineer (into artifacts) like powershell DSC which are used to check the networks/volumes etc. and alert or revert changes if any unauthorised change is detected.

I understand as part of the teams tooling k8, nginx, v-sphere, hyper-v is supported.

If aspire supported network configurations this would allow the continued tooling to work; I also understand that the teams for a number of customer account use Azure Arc in a multi-tenanted type scenario where a tenant is a department of the larger customer account.

I hope that give you a better understanding of where and why aspire network configuration would/could help. I'd be happy to outline more if needed via DM

[davidfowl]

This seems unrelated to supporting docker networks, but maybe I'm missing something.

[ejsmith]

@davidfowl the test container stuff is really nice. What is the story with Aspire supporting integration tests?

[ejsmith]

Just found a sample of using Aspire for tests: https://github.com/dotnet/eShop/blob/main/tests/Ordering.FunctionalTests/OrderingApiFixture.cs

[DamianEdwards]

@ejsmith better support for integration testing is something we're tracking in #1704

[joperezr]

FYI @mitchdenny for thoughts around integration with AppModel

[mitchdenny]

Not something we are tackling for GA at this point.

[3GDXC]

We shouldn't add support for docker networks but we should make a default network so that containers can reference each other by host name.

I'm interested as to why network support is not getting considered? can you enlighten us? maybe the community could do some support if it's simply not a priority?

[davidfowl]

We're not sure what to implement exactly. If you go directly through the docker network, then you circumvent the networking of our orchestrator which means it isn't modeled and doesn't show up anywhere. Projects don't run as containers so you can't attach projects to a container network. Right now, container to container networking is via the host and the orchestrator (dcp) manages proxies to make things like replicas work (see https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/networking-overview for more details). Service discovery might just work but we haven't considered what it means to build the end to end such that if you turned your project into a container and added it to the app model that service discovery would continue to work as is (the follow on implications of those changes etc).

I think we will need to support this, but there has to be experimentation in how to expose this in the app model. I believe there's support for it in the underlying orchestrator, but it's not exposed.

TL;DR: We want to do this, but have decided this isn't a priority for GA so no time has been spent exploring or fleshing out potential solutions.

[EwoutdBoer]

Would be nice to have Elastc search as a component. Then you don't need to use the Elastic tutorial and you also don't run in the 'separate network' issue

[ejsmith]

I started working on Elasticsearch support here: https://github.com/exceptionless/Exceptionless/blob/feature/aspire/src/Exceptionless.AppHost/ElasticsearchExtensions.cs

[davidfowl]

Moving this up the priority list

[mitchdenny]

I think we need to start getting concrete about what features we want here and what the benefits are going to be both in terms of local development, and what it means to deployment scenarios.

Are we just looking to provide a mechanism to fine tune the Docker network configuration? Or do we want to have a fully fletched network resource that we put in the manifest which deployment tools then need to interpret.

I think we are going to run into a translation problem between the network topology that we can represent in Aspire locally and what the deployment tools can represent.

For example, lets say that we want to have multiple Docker networks defined and used in a single Aspire. API TBD, but it might look something like this:

var net1 = builder.AddNetwork("net1");
var net2 = builder.AddNetwork("net2");

var app1 = builder.AddContainer("app1", "image1")
                  .AttachTo(net1);

var app2 = builder.AddContainer("app2", "image2")
                  .AttachTo(net1)
                  .AttachTo(net2);

var app2 = builder.AddContainer("app3", "image3")
                  .AttachTo(net2);

This would result in DCP creating two networks, and when we attach the containers to them. But what is the benefit here for local development?

The above is also problematic for deployment scenarios because target deployment environments won't always be able to faithfully represent a complex network topology like this. For example when we deploy with AZD we deploy all the containers on the same container app environment. Would using networks cause us to create multiple container app environments to ensure network isolation?

... we couldn't represent a container being on two networks so I guess that would have to be an error.

[mitchdenny]

There is a second perspective on this in that we are just using networks for local development to exert more control over address/hostname mapping. We could put each container in its own network and then explicitly map connections between the networks where we know a connection exists in the app model.

[davidfowl]

Can we just create a singlet network for all containers? We want to go through the dcp proxy for them as well for container -> host comms

[mitchdenny]

I'm fine with that. I'm just trying to draw the perimeter on what me mean when we say that we support Docker networks ;)

[mitchdenny]

Moving to backlog for now. We need to do some more design around this and understand the implications in the context of developer edge to cloud as well.

[JonathanAtCenterEdge]

Wanted to chime in as an engineer researching what migration would look like for our team,

Our developers have been utilizing docker-compose for the last 5-ish years but we're desiring to migrate to Aspire. We use an external docker-compose network so that our docker-compose files that are specific to each of our services that reside in individual service-specific repositories can all communicate to each other.

Migration for us from docker-compose to Aspire would require the ability to spin up containers in Aspire with the ability to overload the network until we have all of our services on Aspire, only then could we switch to a theoretical "global" Aspire docker network.

We're very interested in Aspire as it'd give us much easier control over how we spin up our services locally for development purposes.

[davidfowl]

We've made a change to the upcoming version of aspire to support container to container networking with the default aspire-network (#5628). This isn't extensible as we're working towards a bigger network modeling change.

This should help with migration from docker-compose and it will also improve building resources where multiple containers talk to each other.

[3GDXC]

We've made a change to the upcoming version of aspire to support container to container networking with the default aspire-network (#5628). This isn't extensible as we're working towards a bigger network modeling change.

This should help with migration from docker-compose and it will also improve building resources where multiple containers talk to each other.

that's fantastic to here; is there any visibility of the bigger network modeling change? IMHO, it would be very interesting to have the team do rounds of open standup/discussion like the early day of .NET CORE ;) always keen to here background discussion about direction and choice that need to be agreed/disagreed etc.