Ship your apps the GitOps way!
Shipper is a CLI tool that automates deploying new versions of containerized applications to GitOps repository managed by tools such as ArgoCD. Deploying such applications usually means updating a configuration file in a Git repository and writing the newly built container image tag. This file is then read by the CD tool, processed using a templater and then synchronized with a remote infrastructure environment where the application is actually deployed.
A practical example in Neosperience: applications deployed on instances of the Karavel Container Platform, which uses ArgoCD to monitor deployment configurations. To deploy a new image tag, a commit must be performed against either a Helm values.yaml file, or a Kustomize kustomization.yaml file setting a YAML key to the new value.
Shipper automates this by leveraging the native Git provider API (e.g. GitLab, GitHub, BitBucket, etc...) to atomically update the configuration file.
- Azure DevOps
- BitBucket (only BitBucket cloud ie. bitbucket.org)
- Gitea
- GitHub (both GitHub.com and GitHub Enterprise Server)
- GitLab (both self-managed and gitlab.com)
NAME:
shipper - A new cli application
USAGE:
shipper [global options] command [command options] [arguments...]
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--templater value, -p value Template system (available: "helm", "kustomize", "json") [$SHIPPER_PROVIDER]
--repo-kind value, -t value Repository type (available: "gitlab", "github", "gitea", "bitbucket-cloud", "azure") (default: "gitlab") [$SHIPPER_REPO_KIND]
--repo-branch value, -b value Repository branch [$SHIPPER_REPO_BRANCH]
--commit-author value, -a value Commit author in "name <email>" format (default: "Shipper agent <[email protected]>") [$SHIPPER_COMMIT_AUTHOR]
--commit-message value, -m value Commit message (default: "Deploy") [$SHIPPER_COMMIT_MESSAGE]
--container-image value, --ci value Container image [$SHIPPER_CONTAINER_IMAGE, $SHIPPER_CONTAINER_IMAGES]
--container-tag value, --ct value Container tag [$SHIPPER_CONTAINER_TAG, $SHIPPER_CONTAINER_TAGS]
--no-verify-tls If provided, skip X.509 certificate validation on HTTPS requests (default: false) [$SHIPPER_NO_VERIFY_TLS]
--helm-values-file value, --hpath value [helm] Path to values.yaml file [$SHIPPER_HELM_VALUES_FILE, $SHIPPER_HELM_VALUES_FILES]
--helm-image-path value, --himg value [helm] Container image path (default: "image.repository") [$SHIPPER_HELM_IMAGE_PATH, $SHIPPER_HELM_IMAGE_PATHS]
--helm-tag-path value, --htag value [helm] Container tag path (default: "image.tag") [$SHIPPER_HELM_TAG_PATH, $SHIPPER_HELM_TAG_PATHS]
--kustomize-file value, --kfile value [kustomize] Path to kustomization.yaml file [$SHIPPER_KUSTOMIZE_FILE, $SHIPPER_KUSTOMIZE_FILES]
--json-file value, --jfile value [json] Path to JSON file [$SHIPPER_JSON_FILE, $SHIPPER_JSON_FILES]
--gitlab-endpoint value, --gl-uri value [gitlab] Gitlab API endpoint, including "/api/v4" (default: "https://gitlab.com/api/v4") [$SHIPPER_GITLAB_ENDPOINT]
--gitlab-key value, --gl-key value [gitlab] A valid API key with commit access [$SHIPPER_GITLAB_KEY]
--gitlab-project value, --gl-pid value [gitlab] Project ID in "org/project" format [$SHIPPER_GITLAB_PROJECT]
--github-endpoint value, --gh-uri value [github] GitHub API endpoint (include "/api/v3" if using Enterprise Server) (default: "https://api.github.com") [$SHIPPER_GITHUB_ENDPOINT]
--github-key value, --gh-key value [github] Username/password pair in "username:password" format (use a personal access token!) [$SHIPPER_GITHUB_KEY]
--github-project value, --gh-pid value [github] Project ID in "org/project" format [$SHIPPER_GITHUB_PROJECT]
--gitea-endpoint value, --ge-uri value [gitea] Gitea API endpoint (include "/api/v1") [$SHIPPER_GITEA_ENDPOINT]
--gitea-key value, --ge-key value [gitea] Username/application token pair in "username:token" format [$SHIPPER_GITEA_KEY]
--gitea-project value, --ge-pid value [gitea] Project ID in "org/project" format [$SHIPPER_GITEA_PROJECT]
--bitbucket-key value, --bb-key value [bitbucket-cloud] Username/password pair in "username:password" format (use app passwords!) [$SHIPPER_GITLAB_KEY]
--bitbucket-project value, --bb-pid value [bitbucket-cloud] Project path in "org/project" format [$SHIPPER_GITLAB_PROJECT]
--azure-project-id value, --az-pid value [azure-devops] Organization and Project ID, in "org/project" format [$SHIPPER_AZURE_PROJECT_ID]
--azure-repository-id value, --az-rid value [azure-devops] Repository ID (if unsure, use the project ID) [$SHIPPER_AZURE_REPOSITORY_ID]
--azure-key value, --az-key value [azure-devops] Username/application token pair in "username:token" format [$SHIPPER_AZURE_KEY]
--help, -h show help (default: false)
The main use-case for Shipper is to be used as a CI pipeline step. In container-based CI systems like GitLab CI, GitHub Actions and alike, you can run the official container image in a step and invoke shipper with the appropriate flags.
Examples for various CI systems can be found in the examples folder. Please feel free to submit new examples if you have them!
Command line arguments --container-image, --container-tag, --helm-values-file, --helm-image-path, --helm-tag-path and --kustomize-file can be specified multiple times to update multiple images at once. If using the environment variables, these values are comma-separated (eg. CONTAINER_IMAGES=image1,image2).
This is an example of updating multiple images at once:
shipper -p helm --helm-values-file helm/values.yml \
--helm-image-path image.repository --helm-tag-path image.tag --container-image img1 --container-tag tag1 \
--helm-image-path image2.repository --helm-tag-path image2.tag --container-image img2 --container-tag tag2 \
... \
--repo-branch master --commit-author "Test" ...You can mix & match which to specify once and which to specify many times, but every tag must be specified either 1 or N times, with some exceptions where closely-related tags must always be both specified the same amount of times, for example:
- ✔️ one
--helm-values-filebut many--helm-image-path(all changes will be applied to the same YAML file) - ✔️ many
--helm-values-fileand--helm-image-path(every change will be applied to a specific file, the same file can be specified multiple times) - ❌ 3 instances of
--helm-image-pathbut 2 instances of--helm-values-file - ❌ non-equal amount of
--container-image,--container-tag,--helm-image-path,--helm-tag-path
The Helm templater is meant to be used on Helm values.yaml files. For each image to update, you must specify a value for each of the following:
--helm-values-file/SHIPPER_HELM_VALUES_FILES: Path to the Helm values.yaml file to modify--helm-image-path/SHIPPER_HELM_IMAGE_PATHS: YAML field in the Helm values.yaml file to modify with the specified image repository, using dot notation for nested fields (eg.image.repository)--helm-tag-path/SHIPPER_HELM_TAG_PATHS: YAML field in the Helm values.yaml file to modify with the specified image tag, using dot notation for nested fields (eg.image.tag)
The Kustomize templater is meant to be used on kustomization.yaml files using the images list format like in this example.
To use it, you must specify the following, either once or for each image/tag pair:
--kustomize-file/SHIPPER_KUSTOMIZE_FILES: Path to the kustomization.yaml file to modify
The JSON templater is meant for flat JSON files such as CDK context files (cdk.json).
When using JSON, the --container-image argument will be used as the key to update in the JSON file, while the --container-tag argument will be used as the value. You will also need to specify the following value, either once or for each image/tag pair:
--json-file/SHIPPER_JSON_FILES: Path to the JSON file to modify
- Only Git repositories are supported, not Team Foundation (TFVC).
- When creating a personal access token for shipper, only the "Code (Read & write)" permission is needed.
- The author string MUST be in the
John Doe <[email protected]>format or the commit will ignore it and use the default (i.e. the credentials' owner). - You will need both a Project ID (in
org/projectformat) and a Repository ID, if you don't know what your Repository ID is, it's probably the Project ID (without the organization). E.g. If your Project ID ismy-org/my-projectand you have only one repository, your Repository ID ismy-project.
- When creating an app password for shipper, only the permissions
repository:readandrepository:writeare needed. - The author string MUST be in the
John Doe <[email protected]>format or the commit will fail. - Bitbucket cloud integration only works with Bitbucket cloud, Bitbucket server has completely different APIs.
- Due to how the Commit API is implemented, calling shipper with multiple files will result in a multiple commits, one per modified file.
- When creating a personal access token for shipper, only the permissions
repois needed. - The author string MUST be in the
John Doe <[email protected]>format or the commit will fail. - The GitHub Cloud API endpoint is
https://api.github.com, however GitHub Enterprise Server will have something more akin tohttps://HOSTNAME/api/v3 - Due to how the Commit API is implemented, calling shipper with multiple files will result in a multiple commits, one per modified file.
- When creating a project access token for shipper, only the permission
apiis needed. (Role depends on your branch permissions, eg. protected branches)
Found a bug or need a new feature? Open an issue and discuss it with the team!. Don't forget to check out our contribution guide for more information!
See our release guide
Copyright 2022 Neosperience S.P.A.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.