Please rethink how Docker runtime arguments are stored

[SidShetye]

Docker runtime parameters are currently stored in the DockerfileRunArguments

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <DockerfileRunArguments>-e VAR1=VALUE1 -e VAR2=VALUE2 -p 5000:5000 --cap-add=SYS_PTRACE -v /this:/that -p1234:5678 -e BYNOWTHISISOVER1000CHARS=TRUE</DockerfileRunArguments>
  </PropertyGroup>

This is limiting because project has only a single csproj file hence single <DockerfileRunArguments>. This forces people to constantly edit that file to provide parameters based on different scenarios. The natural place seems to be launchSettings.json.

Moving to that launchSettings.json, it does have a Docker profile with commandLineArgs and environmentVariables but they do not work well.

{
  "$schema": "http://json.schemastore.org/launchsettings.json",
  "profiles": {
    "Docker": {
      "commandName": "Docker",
      "commandLineArgs": "THIS",
      "environmentVariables": {
        // AND THIS
      }
    }
  }
}

By that I mean

1. Why have two places to define whats essentially the same user concept? Having two ways raises more questions like
2. What's the order of precedence between commandLineArgs vs <DockerfileRunArguments> - ?
3. What's the order of precedence between environmentVariables vs <DockerfileRunArguments>? I've seen variable in environmentVariables sometimes get ignored.

My thoughts are allowing people to directly express the docker run arguments (instead of an unnecessary and unreliable abstraction like environmentVariables). In fact I'd recommend deprecating <DockerfileRunArguments> and having the entire run arguments in commandLineArgs. Additional creature comforts could be supporting line breaks so it's not an unreadable mess of hundreds of characters but for starters just having a single runtime arg that works across various profiles would be great.

On a related note, even the profile name "Docker": { ... } cannot be renamed to something like Docker PreProd etc without breaking things.

Expand for tools version

Microsoft Visual Studio Community 2019
Version 16.5.0
VisualStudio.16.Release/16.5.0+29911.84
Microsoft .NET Framework
Version 4.8.03752

Installed Version: Community

ASP.NET and Web Tools 2019   16.5.236.49856
ASP.NET and Web Tools 2019

ASP.NET Web Frameworks and Tools 2019   16.5.236.49856
For additional information, visit https://www.asp.net/

Azure App Service Tools v3.0.0   16.5.236.49856
Azure App Service Tools v3.0.0

Azure Functions and Web Jobs Tools   16.5.236.49856
Azure Functions and Web Jobs Tools

C# Tools   3.5.0-beta4-20153-05+20b9af913f1b8ce0a62f72bea9e75e4aa3cf6b0e
C# components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Common Azure Tools   1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Fabric.DiagnosticEvents   1.0
Fabric Diagnostic Events

IntelliCode Extension   1.0
IntelliCode Visual Studio Extension Detailed Info

Microsoft Azure Service Fabric Tools for Visual Studio   16.0
Microsoft Azure Service Fabric Tools for Visual Studio

Microsoft Azure Tools   2.9
Microsoft Azure Tools for Microsoft Visual Studio 2019 - v2.9.30207.1

Microsoft Continuous Delivery Tools for Visual Studio   0.4
Simplifying the configuration of Azure DevOps pipelines from within the Visual Studio IDE.

Microsoft JVM Debugger   1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft Library Manager   2.1.25+gdacdb9b7a1
Install client-side libraries easily to any web project

Microsoft MI-Based Debugger   1.0
Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual Studio Tools for Containers   1.1
Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container.

NuGet Package Manager   5.5.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit https://docs.nuget.org/

ProjectServicesPackage Extension   1.0
ProjectServicesPackage Visual Studio Extension Detailed Info

SQL Server Data Tools   16.0.62003.05170
Microsoft SQL Server Data Tools

TypeScript Tools   16.0.20225.2001
TypeScript Tools for Microsoft Visual Studio

Visual Basic Tools   3.5.0-beta4-20153-05+20b9af913f1b8ce0a62f72bea9e75e4aa3cf6b0e
Visual Basic components used in the IDE. Depending on your project type and settings, a different version of the compiler may be used.

Visual F# Tools 10.8.0.0 for F# 4.7   16.5.0-beta.20104.8+7c4de19faf36647c1ef700e655a52350840c6f03
Microsoft Visual F# Tools 10.8.0.0 for F# 4.7

Visual Studio Code Debug Adapter Host Package   1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

Visual Studio Container Tools Extensions (Preview)   1.0
View, manage, and diagnose containers within Visual Studio.

Visual Studio Tools for Containers   1.0
Visual Studio Tools for Containers

Visual Studio Tools for Kubernetes   1.0
Visual Studio Tools for Kubernetes

[NCarlsonMSFT]

I've added a User Story to our backlog to move our debugging customization to launch profiles rather than being project properties.

For the interim you can use project configurations and property conditions to allow for storing multiple sets of settings and switching between them in the UI without editing the project.

In the meantime to answer some of your questions:

• Once supported the launch profile will have precedence over the project properties.
• commandLineArgs actually would corresponds to DockerDebuggeeArguments (it's the arguments for the service process, not the docker run command); but commandLineArgs is not currently supported.
• environmentVariables specifies the environment for the process being debugged, the environment variables in DockerfileRunArguments. environmentVariables should take precedence over DockerfileRunArguments. If you are seeing variables being ignored please open an issue with repro steps so we can investigate.

For your related note: @pratiksanglikar I believe you opened a bug on the space in the name issue. Do you know what the timeline for that fix being released is?

Lastly, although we are planning to add support for debugging customization to the launch profile. I am not sure if the run arguments will make the cut. For performance reasons we reuse the existing container across debugging sessions as much as possible. Any change to the run arguments would invalidate it and slow down F5. What exactly are you needing to frequently change in these arguments? Like the environment variables, it may be possible to support the customization w/o restarting the container.

[pratiksanglikar]

Hi @NCarlsonMSFT, the space in the name issue is still active. My best guess is that it will go in 16.6 P3.

[SidShetye]

@NCarlsonMSFT Thanks for the clarification on commandLineArgs.

Since you've got this as a user story, could you please share the template proposed for launchSettings.json ? Regardless of how the pieces are expressed in the launchSettings.json, I guess everything needs to be assembled as

debugger debuggerargs -- runbinary runbinaryargs

which is then executed?

[NCarlsonMSFT]

Current proposal would be to mirror the launch profile for .Net Core locally. It would look something like:

"Docker": {
  "commandName": "Docker",
  "launchBrowser": true,
  "launchUrl": "{Scheme}://{ServiceHost}:{ServicePort}",
  "publishAllPorts": true,
  "useSSL": true,
  "executablePath": "dotnet",
  "commandLineArgs": "/app/bin/Debug/netcoreapp3.1/WebApplication1.dll",
  "workingDirectory": "/app",
  "environmentVariables": {
    "ASPNETCORE_ENVIRONMENT": "Development"
  }
}

executablePath would supersede DockerDebuggeeProgram
commandLineArgs would supersede DockerDebuggeeArguments
workingDirectory would supersede DockerDebuggeeWorkingDirectory

[SidShetye]

Thanks but it's still unclear. Lets assume two cases shown below - could you post their corresponding launchSettings.json entries?

1. Single docker command, multiple docker args

How does one express a docker argument list like

-e VAR1=VALUE1 -e VAR2=VALUE2 -p 5000:5000 --cap-add=SYS_PTRACE -v /host/volume/here:/container/volume/there --net my-backendnetwork -p1234:5678 -e BYNOWTHISISOVER1000CHARS=TRUE

Doesn't seem like commandLineArgs since you're expecting dotnet there. Similarly commandLineArgs seems to expect dotnet args.

2. Multiple docker commands, multiple docker args each

Starting an app container on multiple docker networks requires it to be broken into multiple stages like docker create then 1+ docker network connect and a final docker start (documentation) on that app container, instead of a single familiar docker run. Since you'll be debugging non-production environments in VS, it's single container equivalent would look like below. The LB, 2x networks and DB containers would already exist, we'd just want to spin up and debug the app container in the middle in VS.

[ 1x Load  ]___(front network)___[ 1x App container ]___(back n/w)___[ 1x db container ]
[ Balancer ]                     [   (debug this)   ]

How would it's launchSettings.json entry look?

I'm concerned that the Docker CLI is far too expressive and launchSettings.json could be struggling to capture that in it's own restrictive schema. Perhaps allow users to directly embed the expected docker CLI string(s) as-is?

[NCarlsonMSFT]

@SidShetye
The second case would need to be investigated as a separate issue. We don't currently support anything more complex then docker run in the single container tools. Would you be able to configure a compose file to get the configuration you want? If so have you considered using the compose tools?

For the first case:

• the -e arguments can be handled in the environmentVariables
• the -p is only needed if you are doing service to service communication (which based on your second case it seems you are). Is this something you are changing frequently? If not, it seems best to just leave it in the DockerfileRunArguments for now.
  • FYI If you are not doing service to service communication you can remove the -p arguments and use the "publishAllPorts": true, "useSSL": true in your launch profile to let VS and docker coordinate on auto-assigning the exposed ports.
• the other arguments are not planned to be supported elsewhere. Are they something you are changing frequently? If so can you please explain the scenarios?

[SidShetye]

@NCarlsonMSFT : Regarding the complex case of multiple docker commands, multiple args: A compose file would actually be better but we're unsure about the road-map since Docker's docker-compose is to be replaced by docker stack. docker stack supposedly uses the same underlying yml file but only supports some command and doesn't support image building. Since you're building the tools, if you have greater insight, please enlighten!

Regarding the simpler case, I can appreciate that you're trying to break everything down into the existing launchSettings.json but it's going to be painful ...

1. It adds unnecessary burden on your users by having them translate parameters from the docker CLI into some format expected by launchSettings.json. e.g. publishAllPorts and useSSL are pointless artifacts that have no big-picture purpose outside launchSettings.json. Unlike docker CLI args which is relevant and better known outside VS and in some cases even in large scale production (e.g. azure itself has azure app service linux which directly uses docker run).
2. It may not support some basic cases. e.g. From the sounds of it, you don't support mounting volumes (which today I can do via DockerfileRunArguments - even if it's ugly). For us volume mounting is an essential to maintain state.
3. You and your users are constantly playing a frustrating cat-and-mouse between launchSettings.json and the docker CLI as docker's CLI evolves (and it's evolving).

Why won't you support allowing users to simply provide you the docker CLI string they want for a given launchSettings.json profile/session? Tools should simplify, not impose additional burden.

[BigMorty]

@SidShetye, I would use Docker Compose for your scenario. I talk with the Docker Inc folks often and Compose is not going away!

[ncoussemacq]

I fully support this request.

I'm currently working on a project where I need to provide some environment variables to my containers when starting it (typically using -e argument in docker run command line), but some of these parameters are connection strings, which means that I don't want these values to be pushed to source control.
Having these parameters in launchsettings.json rather than in csproj will make sure that I can easily tweak the values when debugging locally without being at risk to push these changes to the source control by mistake (launchsettings.json beeing ignored by git on my side).

[NCarlsonMSFT]

@ncoussemacq setting Environment Variables for the debuggee using the launchsetting's environmentVariables property is already supported today.

[Morgma]

@ncoussemacq setting Environment Variables for the debuggee using the launchsetting's environmentVariables property is already supported today.

It would be nice to be able to reference multiple .env file paths in launchSettings.json, similar to Docker Compose, instead of setting individual environment values, then setting those again in Docker Compose, and separately for the Project launch profile, etc. Passing the "--env-file" arg of the docker run command and would allow a single .env file format across debug (project, Docker, DockerCompose) and also directly compatible with helm/k8s configmap and secret env mounts. Actually, project debugging doesn't support this either, but I make use of DotNetEnv to import my env files into the project debug runtime.

We're going to use the <DockerDebuggeeArguments> property in .csproj to override, as suggested here but it'd be better to set this clearly in the launch settings.

EDIT: We found that <DockerfileRunEnvironmentFiles> may be a better suited workaround