diff --git a/README.md b/README.md index 3925fc33c72..0005910f53e 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ Please refer to the [install guide](https://docs.microsoft.com/cli/azure/install A list of common install issues and their resolutions are available at [install troubleshooting](https://github.com/Azure/azure-cli/blob/dev/doc/install_troubleshooting.md). -### Developer Installation (see below) +### Developer installation (see below) - [Docker](#docker) - [Edge Builds](#edge-builds) @@ -47,7 +47,7 @@ Here are a few features and concepts that can help you get the most out of the A The following examples are showing using the `--output table` format, you can change your default using the `az configure` command. -#### Tab Completion +#### Tab completion We support tab-completion for groups, commands, and some parameters @@ -76,7 +76,8 @@ demo32111vm Windows dcos-master-39DB807E-0 Linux ``` -#### Exit Codes +#### Exit codes + For scripting purposes, we output certain exit codes for differing scenarios. |Exit Code |Scenario | @@ -86,10 +87,23 @@ For scripting purposes, we output certain exit codes for differing scenarios. |2 |Parser error; check input to command line. | |3 |Missing ARM resource; used for existence check from `show` commands. | -#### More Samples and Snippets -For more usage examples, take a look at our [GitHub samples repo](http://github.com/Azure/azure-cli-samples) or [https://docs.microsoft.com/cli/azure/overview](https://docs.microsoft.com/cli/azure/overview). +### Common scenarios and use Azure CLI effectively + +Please check [Tips for using Azure CLI effectively](doc/use_cli_effectively.md). It describes some common scenarios: -For how to use CLI effectively, check out [tips](./doc/use_cli_effectively.md). +- [Output formatting (json, table, or tsv)](doc/use_cli_effectively.md#output-formatting-json-table-or-tsv) +- [Pass values from one command to another](doc/use_cli_effectively.md#pass-values-from-one-command-to-another) +- [Async operations](doc/use_cli_effectively.md#async-operations) +- [Generic update arguments](doc/use_cli_effectively.md#generic-update-arguments) +- [Generic resource commands - `az resource`](doc/use_cli_effectively.md#generic-resource-commands---az-resource) +- [REST API command - `az rest`](doc/use_cli_effectively.md#rest-api-command---az-rest) +- [Quoting issues](doc/use_cli_effectively.md#quoting-issues) +- [Work behind a proxy](doc/use_cli_effectively.md#work-behind-a-proxy) +- [Concurrent builds](doc/use_cli_effectively.md#concurrent-builds) + +### More samples and snippets + +For more usage examples, take a look at our [GitHub samples repo](http://github.com/Azure/azure-cli-samples) or [https://docs.microsoft.com/cli/azure/overview](https://docs.microsoft.com/cli/azure/overview). ## Reporting issues and feedback @@ -97,7 +111,7 @@ If you encounter any bugs with the tool please file an issue in the [Issues](htt To provide feedback from the command line, try the `az feedback` command. -## Developer Installation +## Developer installation ### Docker @@ -114,7 +128,7 @@ For example: $ docker run -u $(id -u):$(id -g) -v ${HOME}:/home/az -e HOME=/home/az --rm -it azuresdk/azure-cli-python:dev ``` -### Edge Builds +### Edge builds If you want to get the latest build from the `dev` branch, you can use our "edge" builds. @@ -154,7 +168,7 @@ If you would like to get builds of arbitrary commit or PR, see: [Try new features before release](doc/try_new_features_before_release.md) -## Developer Setup +## Developer setup If you would like to setup a development environment and contribute to the CLI, see: @@ -162,7 +176,7 @@ If you would like to setup a development environment and contribute to the CLI, [Authoring Command Modules](https://github.com/Azure/azure-cli/tree/dev/doc/authoring_command_modules) -## Contribute Code +## Contribute code This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). diff --git a/doc/use_cli_effectively.md b/doc/use_cli_effectively.md index 39d1bf28169..efbd4a53adf 100644 --- a/doc/use_cli_effectively.md +++ b/doc/use_cli_effectively.md @@ -1,8 +1,8 @@ -# Tips for using Azure CLI effectively # +# Tips for using Azure CLI effectively For clarity, Bash scripts are used inline. Windows batch or PowerScript examples are listed in the appendix, which you can use to build similar examples. -## Use the right output mode for your work (json, table, or tsv) ## +## Output formatting (json, table, or tsv) 1. `json` format is the CLI's default, and is intended to give you the most comprehensive information. If you prefer a different format, use the `--output` argument to override for an individual command invocation, or use `az configure` to update your global default. Note that JSON format preserves the double quotes, generally making in unsuitable for scripting purposes. @@ -22,7 +22,7 @@ For clarity, Bash scripts are used inline. Windows batch or PowerScript examples az vm stop --ids $vm_ids ``` -## Passing values from one command to the other ## +## Pass values from one command to another 1. If the value will be used more than once, assign it to a variable. Note the use of `-o tsv` in the following example: @@ -58,7 +58,7 @@ For clarity, Bash scripts are used inline. Windows batch or PowerScript examples az vm list -d -g my_rg --query "[?powerState=='VM stopped'].id" -o tsv | az vm start --ids @- ``` -## Async Operations ## +## Async operations Many commands and group expose `--no-wait` flags on their long-running operations as well as a dedicated `wait` command. These become handy for certain scenarios: @@ -78,7 +78,8 @@ Many commands and group expose `--no-wait` flags on their long-running operation az vm wait --created --ids $vm1_id $vm2_id ``` -## Using the Generic Update Arguments ## +## Generic update arguments + Most update commands in the CLI feature the three generic arguments: `--add`, `--set` and `--remove`. These arguments are powerful but often less convenient than the strongly-typed arguments typically featured in update commands. The CLI provides strongly-typed arguments for most common scenarios for ease-of-use, but if the property you want to set isn't listed, the generic update arguments will often present a path forward to unblock you without having to wait for a new release. 1. The generic update syntax isn't the most user friendly, so it will require some patience. @@ -94,20 +95,53 @@ Most update commands in the CLI feature the three generic arguments: `--add`, `- az vm update -g my_rg -n my_vm --add storageProfile.dataDisks @~/my_disk.json ``` -## Quoting Issues ## +## Generic resource commands - `az resource` + +There may be cases where a service you are interested in does not have CLI command coverage. You can use the `az resource create/show/list/delete/update/invoke-action` commands to work with these resources. Here are a few suggestions: +1. If only `create/update` are involved, consider using `az group deployment create`. Leverage [Azure Quickstart Templates](https://github.com/Azure/azure-quickstart-templates) for working examples. +2. Check out the Rest API reference for the request payload, URL and API version. As an example, check out the community's comments on [how to create AppInsights](https://github.com/Azure/azure-cli/issues/5543). + +## REST API command - `az rest` + +If neither generic update arguments nor `az resource` meets your needs, you can use `az rest` command to call the REST API. It automatically authenticates using the logged-in credential and sets header `Content-Type: application/json`. + +This is extremely useful for calling [Microsoft Graph API](https://docs.microsoft.com/en-us/graph/api/overview?toc=./ref/toc.json&view=graph-rest-1.0) which is not currently supported by CLI commands ([#12946](https://github.com/Azure/azure-cli/issues/12946)). + +For example, to update `redirectUris` for an [Application](https://docs.microsoft.com/en-us/graph/api/resources/application?view=graph-rest-1.0), we call the [Update application](https://docs.microsoft.com/en-us/graph/api/application-update?view=graph-rest-1.0&tabs=http) REST API with: + +```sh +# Line breaks for legibility only + +# Get the application +az rest --method GET + --url 'https://graph.microsoft.com/v1.0/applications/b4e4d2ab-e2cb-45d5-a31a-98eb3f364001' + +# Update `redirectUris` for `web` property +az rest --method PATCH + --url 'https://graph.microsoft.com/v1.0/applications/b4e4d2ab-e2cb-45d5-a31a-98eb3f364001' + --body '{"web":{"redirectUris":["https://myapp.com"]}}' +``` + +## Quoting issues + +This becomes an issue because when the command shell (Bash, Zsh, Windows Command Prompt, PowerShell, etc) parses the CLI command, it will interpret the quotes and spaces. Always refer to the documents when you are uncertain about the usage of a shell: + +- Bash: [Quoting](https://www.gnu.org/software/bash/manual/html_node/Quoting.html) +- PowerShell: [About Quoting Rules](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_quoting_rules) +- Windows Command Prompt: [How-to: Escape Characters, Delimiters and Quotes at the Windows command line](https://ss64.com/nt/syntax-esc.html) -This becomes an issue because when the command shell (bash, zsh, Windows Command Prompt, PowerShell etc) parses the CLI command, it will interpret the quotes. To avoid surprises, here are a few suggestions: +To avoid unanticipated results, here are a few suggestions: 1. If the value contains whitespace, you must wrap it in quotes. 2. In bash or Windows PowerShell, both single and double quotes will be interpreted, while in Windows Command Prompt, only double quotes are handled which means single quotes will be interpreted as a part of the value. -3. If your command only runs on bash (or zsh), using single quotes has the benefit of preserving the content inside. This can be very helpful when supplying inline JSON. For example this works in bash: `'{"foo": "bar"}'` -4. If your command will run on Windows Command Prompt, you must use double quotes exclusively. If the value contains double quotes, you must escape it: "i like to use \\" a lot". The Command Prompt equivalent of the above would be: `"{\"foo\": \"bar\"}"` -5. Exported variables in bash inside double quotes will be evaluated. If this is not what you want, again use \\ to escape it like `"\\$var"` or use single quotes `'$var'`. +3. If your command only runs on Bash (or Zsh), using single quotes has the benefit of preserving the content inside. This can be very helpful when supplying inline JSON. For example this works in bash: `'{"foo": "bar"}'` +4. If your command will run on Windows Command Prompt, you must use double quotes exclusively. If the value contains double quotes, you must escape it: `"i like to use \" a lot"`. The Command Prompt equivalent of the above would be: `"{\"foo\": \"bar\"}"` +5. Exported variables in bash inside double quotes will be evaluated. If this is not what you want, again use `\ ` to escape it like `"\$var"` or use single quotes `'$var'`. 6. A few CLI arguments, including the generic update arguments, take a list of space-separated values, like `= =`. Since the key name and value can take arbitrary string which might contain whitespace, using quotes will be necessary. Wrap the pair, not individual key or value. So `"my name"=john` is wrong. Instead, use `"my name=john"`. For example: ```sh az webapp config appsettings set -g my_rg -n my_web --settings "client id=id1" "my name=john" ``` -7. Use CLI's `@` convention to load from a file so to bypass the shell's intepretion mechanisms: +7. Use CLI's `@` convention to load from a file so to bypass the shell's interpretation mechanisms: ```sh az ad app create --display-name my-native --native-app --required-resource-accesses @manifest.json ``` @@ -115,23 +149,78 @@ This becomes an issue because when the command shell (bash, zsh, Windows Command - `--arg foo bar`: OK. Unquoted, space-separated list - `--arg "foo" "bar"`: OK: Quoted, space-separated list - `--arg "foo bar"`: BAD. This is a string with a space in it, not a space-separated list. -9. When running Azure CLI commands in PowerShell, parsing errors will occur when the arguments contain special characters of PowerShell, such as at `@`. You can solve this problem by adding `` ` `` before the special character to escape it, or by enclosing the argument with single or double quotes `'`/`"`. For example, `az group deployment create --parameters @parameters.json` dose't work in PowerShell because `@` is parsed as a [splatting symbol](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_splatting). To fix this, you may change the argument to `` `@parameters.json`` or `'@parameters.json'`. -10. On Windows, `az` is a batch script (at `C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd`). When there is no space in an argument, PowerShell will strip the quotes and pass the argument to Command Prompt. This causes the argument to be parsed again by Command Prompt. For example, when running `az "a&b"` in PowerShell, `b` is treated as a separate command instead of part of the argument like Command Prompt does, because quotes are removed by PowerShell and the ampersand `&` is parsed again by Command Prompt as a [command separator](https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-xp/bb490954(v=technet.10)#using-multiple-commands-and-conditional-processing-symbols). - - To prevent this, you may use [stop-parsing symbol](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_parsing) `--%` between `az` and arguments like `az --% vm create ...`. This can also solve the above-mentioned special character issue and is recommended whenever any issue happens when invoking a batch script from PowerShell. - - > The stop-parsing symbol (--%), introduced in PowerShell 3.0, directs PowerShell to refrain from interpreting input as PowerShell commands or expressions. - > - > When it encounters a stop-parsing symbol, PowerShell treats the remaining characters in the line as a literal. - - This issue is tracked at https://github.com/PowerShell/PowerShell/issues/1995#issuecomment-539822061 - -## Generic Resource Commands - There may be cases where a service you are interested in does not have CLI command coverage. You can use the `az resource create/show/list/delete/update/invoke-action` commands to work with these resources. A few suggestions here: - 1. If only `create/update` are involved, consider using `az group deployment create`. Leverage [Azure Quickstart Templates](https://github.com/Azure/azure-quickstart-templates) for working examples. - 2. Check out the Rest API reference for the request payload, URL and API version. As an example, check out the community's comments on [how to create AppInsights](https://github.com/Azure/azure-cli/issues/5543). - -## Working behind a proxy +9. When running Azure CLI commands in PowerShell, parsing errors will occur when the arguments contain special characters of PowerShell, such as at `@`. You can solve this problem by adding `` ` `` before the special character to escape it, or by enclosing the argument with single or double quotes `'`/`"`. For example, `az group deployment create --parameters @parameters.json` doesn't work in PowerShell because `@` is parsed as a [splatting symbol](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_splatting). To fix this, you may change the argument to `` `@parameters.json`` or `'@parameters.json'`. +10. When using `--query` with a command, some characters of [JMESPath](https://jmespath.org/specification.html) need to be escaped in the shell. For example, in Bash: + ```sh + # Wrong, as the dash needs to be quoted in a JMESPath query + $ az version --query azure-cli + az version: error: argument --query: invalid jmespath_type value: 'azure-cli' + + # Wrong, as the dash needs to be quoted in a JMESPath query, but quotes are interpreted by Bash + $ az version --query "azure-cli" + az version: error: argument --query: invalid jmespath_type value: 'azure-cli' + + # Correct + $ az version --query '"azure-cli"' + "2.5.1" + + $ az version --query \"azure-cli\" + "2.5.1" + + $ az version --query "\"azure-cli\"" + "2.5.1" + ``` + + In Command Prompt: + ```cmd + > az version --query "\"azure-cli\"" + "2.5.1" + + > az version --query \"azure-cli\" + "2.5.1" + ``` + + In PowerShell (see item 12 for why extra escaping is needed): + ```powershell + > az version --query '\"azure-cli\"' + "2.5.1" + + > az version --query "\`"azure-cli\`"" + "2.5.1" + + > az version --query "\""azure-cli\""" + "2.5.1" + + > az --% version --query "\"azure-cli\"" + "2.5.1" + + > az --% version --query \"azure-cli\" + "2.5.1" + ``` + +11. The best way to troubleshoot a quoting issue is to run the command with `--debug` flag. It reveals the actual arguments received by CLI in [Python's syntax](https://docs.python.org/3/tutorial/introduction.html#strings). For example, in Bash: + + ```sh + # Wrong, as quotes and spaces are interpreted by Bash + $ az {"key": "value"} --debug + Command arguments: ['{key:', 'value}', '--debug'] + + # Wrong, as quotes are interpreted by Bash + $ az {"key":"value"} --debug + Command arguments: ['{key:value}', '--debug'] + + # Correct + $ az '{"key":"value"}' --debug + Command arguments: ['{"key":"value"}', '--debug'] + + # Correct + $ az "{\"key\":\"value\"}" --debug + Command arguments: ['{"key":"value"}', '--debug'] + ``` +12. Due to a known issue of PowerShell, some extra escaping rules apply, see [Quoting issues with PowerShell](quoting-issues-with-powershell.md) for more information + +## Work behind a proxy + Proxy is common behind corporate network or introduced by tracing tools like Fiddler, mitmproxy, etc. If the proxy uses self-signed certificates, the Python [Requests](https://github.com/kennethreitz/requests) library which CLI uses will throw `SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)`. There are 2 ways to handle this error: 1. Set environment variable `REQUESTS_CA_BUNDLE` to the path of CA bundle certificate file in PEM format. This is recommended if you use CLI frequently behind a corporate proxy. The default CA bundle which CLI uses is located at `C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\certifi\cacert.pem` on Windows and ` /opt/az/lib/python3.6/site-packages/certifi/cacert.pem` on Linux. You may append the proxy server's certificate to this file or copy the contents to another certificate file, then set `REQUESTS_CA_BUNDLE` to it. For example: @@ -155,7 +244,9 @@ Proxy is common behind corporate network or introduced by tracing tools like Fid If you are using az on a build machine, and multiple jobs can be run in parallel, then there is a risk that the login tokens are shared between two build jobs is the jobs run as the same OS user. To avoid mix ups like this, set AZURE_CONFIG_DIR to a directory where the login tokens should be stored. It could be a randomly created folder, or just the name of the jenkins workspace, like this ```AZURE_CONFIG_DIR=.``` ## Appendix + ### Windows batch scripts for saving to variables and using it later + ```batch ECHO OFF SETLOCAL diff --git a/src/azure-cli-core/azure/cli/core/util.py b/src/azure-cli-core/azure/cli/core/util.py index 48e5efab5f9..0113842eed1 100644 --- a/src/azure-cli-core/azure/cli/core/util.py +++ b/src/azure-cli-core/azure/cli/core/util.py @@ -31,7 +31,7 @@ SSLERROR_TEMPLATE = ('Certificate verification failed. This typically happens when using Azure CLI behind a proxy ' 'that intercepts traffic with a self-signed certificate. ' # pylint: disable=line-too-long - 'Please add this certificate to the trusted CA bundle: https://github.com/Azure/azure-cli/blob/dev/doc/use_cli_effectively.md#working-behind-a-proxy. ' + 'Please add this certificate to the trusted CA bundle: https://github.com/Azure/azure-cli/blob/dev/doc/use_cli_effectively.md#work-behind-a-proxy. ' 'Error detail: {}') _PROXYID_RE = re.compile(