diff --git a/.editorconfig b/.editorconfig index 3aa6398c7..c7a938d82 100644 --- a/.editorconfig +++ b/.editorconfig @@ -52,7 +52,7 @@ indent_size = 4 [tox.ini] max_line_length = 120 -[Makefile] +[Makefile*] indent_style = tab indent_size = 4 tab_width = 4 diff --git a/.github/ISSUE_TEMPLATE/bug-template.yaml b/.github/ISSUE_TEMPLATE/bug-template.yaml index 04e283e1a..0366d6da2 100644 --- a/.github/ISSUE_TEMPLATE/bug-template.yaml +++ b/.github/ISSUE_TEMPLATE/bug-template.yaml @@ -59,7 +59,7 @@ body: attributes: label: Possible Solution description: | - Suggest a fix/reason for the bug + Suggest a fix/reason for the bug. validations: required: false - type: textarea @@ -70,6 +70,12 @@ body: Anything else that might be relevant for troubleshooting this bug. Providing context helps us come up with a solution that is most useful in the real world. + + If you are facing a build or deployment issue, please include details + about the environment you are working in. You can run the following + command to capture all relevant details in one report: + + make version_report validations: required: false - type: input @@ -79,3 +85,11 @@ body: description: The ADF version you are running. validations: required: true + - type: checkboxes + attributes: + label: Contributing a fix? + description: > + Are you are able to contribute and resolve this issue? + options: + - label: Yes, I am working on a fix to resolve this issue + required: false diff --git a/.github/workflows/adf.yml b/.github/workflows/adf.yml index 8698087ce..e96f43068 100644 --- a/.github/workflows/adf.yml +++ b/.github/workflows/adf.yml @@ -19,10 +19,13 @@ jobs: - name: Set the correct Node version using nvm shell: bash -l {0} run: nvm install - - name: Install dependencies - run: make init + - name: Test compatibility of source dependencies + run: | + make clean src_deps + echo "Source dependencies are compatible!" - name: Run tox - # Run tox using the version of Python in `PATH` run: | - tox --version - tox + make clean tox + - name: Build fully + run: | + make clean build diff --git a/.gitignore b/.gitignore index 8380271e1..a26a7493a 100644 --- a/.gitignore +++ b/.gitignore @@ -18,9 +18,12 @@ master-deploy.yml shared_layer.zip .aws-sam samconfig.toml +samconfig.yml +samconfig.yaml pipeline.json template-sam.yml deploy.sh +Makefile.new # Byte-compiled / optimized / DLL files __pycache__ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0fe9d7ecd..b7d61834e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -16,33 +16,50 @@ features. When filing an issue, please check [existing open](https://github.com/awslabs/aws-deployment-framework/issues), and [recently closed](https://github.com/awslabs/aws-deployment-framework/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20) -issues to make sure somebody else hasn't already reported the issue. +issues to make sure somebody else has not already reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: - A reproducible test case or series of steps - The version of our code being used +- For deployment related bugs, please include the details returns by the + following command as executed on the development machine: + `make report_versions` - Any modifications you have made relevant to the bug - Anything unusual about your environment or deployment ## Running tests locally -In order to run the tests locally you need a virtual environment that is used -by [tox](https://pypi.org/project/tox/). +In order to run the tests locally you need install the development requirements +in the python virtual environment that is used. -1. Create a virtual environment: `virtualenv .tox/env` -2. Activate the virtual environment: `source .tox/env/bin/activate` -3. Install dependencies: `make init` -4. To run the tests, execute: `tox` +To ensure test execute consistently, ADF relies on +[tox](https://pypi.org/project/tox/). + +The easiest way to install and run these tests is to run: + +```bash +make tox +``` + +Alternatively, you can also install the required dependencies in your +python virtual environment by running: + +```bash +pip install -r requirements-dev.txt +``` + +To run the tests, simply execute next: `tox`. +This will create a virtual environment managed by tox to run the tests in. ## Running linters locally You need to have NodeJS and Docker installed on your computer to run MegaLinter locally with MegaLinter Runner. -You can run mega-linter-runner without installation by using `npx` (Run from the -root of the repository!). +You can run mega-linter-runner without installation by using `npx`. +Make sure to execute this from the root of the repository. ```sh npx mega-linter-runner @@ -54,6 +71,13 @@ Some linters can automatically fix findings by running the command below. npx mega-linter-runner --fix ``` +## Deploy your changes + +To deploy your changes, make sure to commit them in your local repository +first. Once committed, follow the [installation +guide](./docs/installation-guide.md) to deploy them into your management +account. + ## Contributing via Pull Requests Contributions via pull requests are much appreciated. @@ -121,6 +145,39 @@ following account ids may be used: - `012345671234` - `123456789012` +## Resolving Build and deployment issues + +Please capture the environment the build/deployment issue occurs in: + +```bash +make version_report +``` + +Before you report an issue, please try again after cleaning the environment: + +```bash +make clean +``` + +Possibly, this issue is fixed already, please update your Makefile and try +again after: + +```bash +make update_makefile +``` + +If that did not resolve the issue, please try running: + +```bash +make clean deps_build +``` + +Or download an older version of the Makefile by running: + +```bash +make UPDATE_VERSION=make/2.0 update_makefile +``` + ## Code of Conduct This project has adopted the diff --git a/Makefile b/Makefile index 1185c8d7d..13b854379 100644 --- a/Makefile +++ b/Makefile @@ -1,28 +1,223 @@ -init: - pip install -r requirements.txt -r requirements-dev.txt - -test: - # Run unit tests - pytest src/lambda_codebase/account -vvv -s -c src/lambda_codebase/account/pytest.ini - pytest src/lambda_codebase/account_processing -vvv -s -c src/lambda_codebase/account_processing/pytest.ini - pytest src/lambda_codebase/initial_commit -vvv -s -c src/lambda_codebase/initial_commit/pytest.ini - pytest src/lambda_codebase/initial_commit/bootstrap_repository -vvv -s -c src/lambda_codebase/initial_commit/bootstrap_repository/pytest.ini - pytest src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase -vvv -s -c src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/pytest.ini - pytest src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/initial_commit -vvv -s -c src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/initial_commit/pytest.ini - pytest src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/determine_default_branch -vvv -s -c src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/determine_default_branch/pytest.ini - pytest src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/helpers -vvv -s -c src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/helpers/pytest.ini - pytest src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/python -vvv -s -c src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/python/pytest.ini - pytest src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk -vvv -s -c src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/cdk/pytest.ini - pytest src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared -vvv -s -c src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/shared/pytest.ini - -lint: - # Linter performs static analysis to catch latent bugs - find src -iname "*.py" -not -path "src/.aws-sam/*" | xargs pylint --verbose --rcfile .pylintrc - find src -iname "*.yml" -o -iname "*.yaml" -not -path "src/.aws-sam/*" | xargs yamllint -c .yamllint.yml - cfn-lint - -build: - sam build -u - -deploy: - sam deploy --capabilities CAPABILITY_NAMED_IAM CAPABILITY_AUTO_EXPAND +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Makefile versions +MAKEFILE_VERSION := 2.0 +UPDATE_VERSION := make/latest + +# Repository versions +SRC_VERSION := $(shell git describe --tags --match 'v[0-9]*') +SRC_VERSION_TAG_ONLY := $(shell git describe --tags --abbrev=0 --match 'v[0-9]*') + +# ADF Related URLs +SRC_URL_BASE := https://github.com/awslabs/aws-deployment-framework +RAW_URL_BASE := https://raw.githubusercontent.com/awslabs/aws-deployment-framework + +UPDATE_URL := "$(RAW_URL_BASE)/$(UPDATE_VERSION)/Makefile" +SRC_TAGGED_URL_BASE := "$(SRC_URL_BASE)/tree/$(SRC_VERSION_TAG_ONLY)" +MAKE_TAGGED_URL_BASE := "$(SRC_URL_BASE)/tree/make/$(MAKEFILE_VERSION)" + +# Command line colors +CLR_RED := $(shell printf "\033[0;31m") +CLR_GREEN := $(shell printf "\033[0;32m") +CLR_YELLOW := $(shell printf "\033[0;33m") +CLR_BLUE := $(shell printf "\033[0;34m") +CLR_END := $(shell printf "\033[0m") + +# Files to work with +SAM_VERSIONED_TEMPLATE := ./src/template-sam.yml +SAM_BUILD_DIR := ./.aws-sam/build +SRC_DIR := ./src + +REQUIREMENTS := $(shell find . -maxdepth 1 -name 'requirements.txt' -or -name 'requirements-dev.txt') +SRC_REQUIREMENTS := $(shell find $(SRC_DIR) -name 'requirements.txt' -or -name 'requirements-dev.txt') +PIP_INSTALL_REQUIREMENTS := $(addprefix -r ,$(REQUIREMENTS)) +PIP_INSTALL_SRC_REQUIREMENTS := $(addprefix -r ,$(SRC_REQUIREMENTS)) + +# Actions + +# Default action should come first, jump to build: +all: build + +# Which actions do not create an actual file like make expects: +.PHONY: all clean update_makefile +.PHONY: report_makefile_version report_versions version_report +.PHONY: build_debs deps src_deps docker version_number git_ignore docs tox +.PHONY: pre_build pre_deps_build sam_build post_build build deps_build +.PHONY: pre_deploy_msg pre_deploy sam_deploy post_deploy deploy + +.venv: .venv/is_ready + +.venv/is_ready: + ( \ + test -d .venv || python3 -m venv .venv; \ + touch .venv/is_ready; \ + ) + +clean: + test -e $(SAM_VERSIONED_TEMPLATE) && rm $(SAM_VERSIONED_TEMPLATE) || exit 0 + test -d $(SAM_BUILD_DIR) && rm -r $(SAM_BUILD_DIR) || exit 0 + test -d .venv && rm -r .venv || exit 0 + +update_makefile: report_makefile_version + @( \ + ( \ + which curl && curl -fsSL $(UPDATE_URL) -o ./Makefile.new \ + ) || ( \ + which wget && wget -q $(UPDATE_URL) -O ./Makefile.new \ + ) || ( \ + echo "$(CLR_RED)No curl or wget, please install and try again$(CLR_END)" && \ + exit 1 \ + ); \ + ) + @echo "Updated Makefile info:" + @make -f ./Makefile.new clean build_deps report_makefile_version + @mv ./Makefile.new ./Makefile + @echo "$(CLR_GREEN)Update complete$(CLR_END)" + +report_makefile_version: + @echo "Makefile: v$(MAKEFILE_VERSION) $$(shasum --algorithm 256 Makefile)" + +report_versions: report_makefile_version + @echo "$(CLR_YELLOW)*** Beginning of ADF Version Report ***$(CLR_END)" + @echo "ADF Source version: $(SRC_VERSION)" + @echo "ADF $$(cat src/template.yml | grep SemanticVersion | xargs)" + @echo "" + @echo "Hardware platform: $$(uname --hardware-platform || echo 'n/a')" + @echo "Kernel name: $$(uname --kernel-name || echo 'n/a')" + @echo "Kernel release: $$(uname --kernel-release || echo 'n/a')" + @echo "" + @test -e /etc/os-release && echo "OS Release:" && cat /etc/os-release + @echo "" + @echo "Disk:" + @df -h $$PWD || echo 'N/A' + @echo "" + @echo "Dependencies:" + @echo "docker: $$(docker --version || echo 'n/a')" + @echo "git: $$(git --version || echo 'n/a')" + @echo "make: $$(make --version || echo 'n/a')" + @( \ + . .venv/bin/activate; \ + pip --version; \ + pip list; \ + ) + @echo "" + git status + @echo "$(CLR_YELLOW)*** End of ADF Version Report ***$(CLR_END)" + +version_report: report_versions + +build_deps: .venv + ( \ + . .venv/bin/activate; \ + pip install aws-sam-cli yq; \ + ) + +deps: .venv + ( \ + . .venv/bin/activate; \ + pip install $(PIP_INSTALL_REQUIREMENTS); \ + ) + +src_deps: .venv + ( \ + . .venv/bin/activate; \ + pip install $(PIP_INSTALL_SRC_REQUIREMENTS); \ + ) + +tox: deps + # Run tests via tox + @( \ + . .venv/bin/activate; \ + tox --version; \ + tox; \ + ) + +docker: + @echo "Prepare docker to support all architectures..." + docker run --rm --privileged multiarch/qemu-user-static --reset -p yes + +version_number: .venv + @echo "Generate a new version number..." + cp $(SRC_DIR)/template.yml $(SAM_VERSIONED_TEMPLATE) + ( \ + . .venv/bin/activate; \ + BASE_ADF_VERSION=$$(cat $(SAM_VERSIONED_TEMPLATE) | yq '.Metadata."AWS::ServerlessRepo::Application".SemanticVersion' -r); \ + COMMIT_ADF_VERSION=$(SRC_VERSION); \ + sed "s/Version: $$BASE_ADF_VERSION/Version: $$COMMIT_ADF_VERSION/g" -i $(SAM_VERSIONED_TEMPLATE); \ + ) + +git_ignore: + mkdir -p $(SAM_BUILD_DIR)/InitialCommitHandler/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/initial_commit/pipelines_repository + cp $(SRC_DIR)/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/initial_commit/pipelines_repository/.gitignore $(SAM_BUILD_DIR)/InitialCommitHandler/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/initial_commit/pipelines_repository/.gitignore + cp $(SRC_DIR)/lambda_codebase/initial_commit/bootstrap_repository/.gitignore $(SAM_BUILD_DIR)/InitialCommitHandler/bootstrap_repository/.gitignore + +docs: + @echo "" + @echo "$(CLR_YELLOW)Please use the guides related to ADF $(SRC_VERSION_TAG_ONLY):$(CLR_END)" + @echo "" + @( \ + echo "$(SRC_VERSION_TAG_ONLY)" | grep -E 'v[0-3]\.' &> /dev/null && \ + echo "* $(CLR_BLUE)$(MAKE_TAGGED_URL_BASE)/docs/installation-guide.md$(CLR_END)" || \ + echo "* $(CLR_BLUE)$(SRC_TAGGED_URL_BASE)/docs/installation-guide.md$(CLR_END)"; \ + ) + @echo "" + @echo "* $(CLR_BLUE)$(SRC_TAGGED_URL_BASE)/docs/admin-guide.md$(CLR_END)" + @echo "" + @echo "* $(CLR_BLUE)$(SRC_TAGGED_URL_BASE)/docs/user-guide.md$(CLR_END)" + @echo "" + +pre_build: build_deps docker version_number git_ignore + +pre_deps_build: deps docker version_number git_ignore + +sam_build: + @( \ + . .venv/bin/activate; \ + sam build \ + --use-container \ + --template $(SAM_VERSIONED_TEMPLATE); \ + ) + +post_build: + @rm $(SAM_VERSIONED_TEMPLATE) + @echo "" + @echo "$(CLR_GREEN)ADF built successfully!$(CLR_END)" + @echo "$(CLR_GREEN)To deploy ADF, please run:$(CLR_END) make deploy" + @echo "" + +build: pre_build sam_build post_build + +deps_build: pre_deps_build sam_build post_build + +pre_deploy_msg: + @echo "" + @echo "" + @echo "$(CLR_GREEN)Thank you for deploying ADF, we are about to proceed$(CLR_END)" + @echo "" + @echo "$(CLR_YELLOW)In the next step, a few questions need to be answered.$(CLR_END)" + @echo "$(CLR_YELLOW)Please use the following guide to answer these:$(CLR_END)" + @echo "" + @( \ + echo "$(SRC_VERSION_TAG_ONLY)" | grep -E 'v[0-3]\.' &> /dev/null && \ + echo "$(CLR_BLUE)$(MAKE_TAGGED_URL_BASE)/docs/installation-guide.md$(CLR_END)" || \ + echo "$(CLR_BLUE)$(SRC_TAGGED_URL_BASE)/docs/installation-guide.md$(CLR_END)"; \ + ) + @echo "" + @echo "" + +pre_deploy: build_deps pre_deploy_msg + +sam_deploy: + @( \ + . .venv/bin/activate; \ + sam deploy \ + --guided \ + --capabilities CAPABILITY_NAMED_IAM CAPABILITY_AUTO_EXPAND \ + --no-disable-rollback \ + --tags "ADF_VERSION=$(SRC_VERSION)"; \ + ) + +post_deploy: docs + +deploy: pre_deploy sam_deploy post_deploy diff --git a/Makefile.tox b/Makefile.tox new file mode 100644 index 000000000..1cffe0b6f --- /dev/null +++ b/Makefile.tox @@ -0,0 +1,24 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Files to work with +SRC_DIR := ./src +TEST_CONFIGS := $(shell find $(SRC_DIR) -name 'pytest.ini') + +all: test lint + +.PHONY: all test lint + +test: + # Run unit tests + ( \ + for config in $(TEST_CONFIGS); do \ + pytest $$(dirname $$config) -vvv -s -c $$config; \ + done \ + ) + +lint: + # Linter performs static analysis to catch latent bugs + find $(SRC_DIR) -iname "*.py" | xargs pylint --verbose --rcfile .pylintrc + find $(SRC_DIR) -iname "*.yml" -o -iname "*.yaml" | xargs yamllint -c .yamllint.yml + cfn-lint diff --git a/README.md b/README.md index 64fcba76f..546fcf2a1 100644 --- a/README.md +++ b/README.md @@ -27,12 +27,14 @@ template as its baseline. ## Quick Start -Launch ADF via the -[Serverless Application Repository](https://console.aws.amazon.com/lambda/home?region=us-east-1#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:112893979820:applications/aws-deployment-framework) -within the AWS Console. - - Refer to the [Installation Guide](docs/installation-guide.md) for Installation steps. -- Refer to the [User Guide](docs/user-guide.md) for using ADF once it is setup. +- Refer to the [Admin Guide](docs/admin-guide.md) on guidance how-to manage and + administrate ADF. +- Refer to the [User Guide](docs/user-guide.md) for using ADF to generate + and manage pipelines. +- Refer to the [Technical Guide](docs/technical-guide.md) if you want to learn + more about the inner workings of ADF. For example in case you want to + contribute or build on top of ADF. - Refer to the [Samples Guide](docs/samples-guide.md) for a detailed walk through of the provided samples. diff --git a/docs/admin-guide.md b/docs/admin-guide.md index 18c60a0f6..4f9b1a5e9 100644 --- a/docs/admin-guide.md +++ b/docs/admin-guide.md @@ -67,11 +67,11 @@ The configuration properties are synced into AWS Systems Manager Parameter Store and are used for certain orchestration options throughout your Organization. -Below is an example of an `adfconfig.yml` file. When you install ADF via the -Serverless Application Repository, some of the information entered at the time -of deployment will be passed into the `adfconfig.yml` that is committed to the -bootstrap repository as a starting point. You can always edit it and push it -back into the bootstrap repository to update any values. +Below is an example of an `adfconfig.yml` file. When you deploy ADF, +the information entered at the time of deployment will be passed into the +`adfconfig.yml` that is committed to the bootstrap repository as a starting +point. You can always edit it and push it back into the bootstrap repository +to update these values. ```yaml roles: @@ -320,7 +320,7 @@ that is moved into a specific Organizational Unit in AWS Organizations. Bootstrapping of AWS accounts is a convenient way to apply a baseline to an account or sub-set of accounts based on the structure of your AWS Organization. -When deploying ADF via the Serverless Application Repository, a CodeCommit +When deploying ADF, a CodeCommit repository titled `aws-deployment-framework-bootstrap` will also be created. This repository acts as an entry point for bootstrapping templates. The definition of which templates are applied to which Organization Unit are defined @@ -473,8 +473,8 @@ template as required, however, the default resources should not be removed. #### Bootstrapping Regions When you setup the initial configuration for the AWS Deployment Framework you -define your parameters in the Serverless Application Repository, some of these -details get placed into the [adfconfig.yml](#adfconfig). +define your initial parameters, a subset of these get placed into the +[adfconfig.yml](#adfconfig). This file defines the regions you will use for not only bootstrapping but which regions will later be used as targets for deployment pipelines. Be sure you read @@ -922,42 +922,21 @@ account in us-east-1. Check the CloudFormation stack output or tag of the - In the outputs tab, it will show the version as the `ADFVersionNumber`. - In the tags on the CloudFormation stack, it is presented as - `serverlessrepo:semanticVersion`. + `ADFVersion` or `serverlessrepo:semanticVersion`. ### Latest ADF version that is available If you want to check which version is the latest one available, go to the -management account in `us-east-1`: - -1. Navigate to the AWS Deployment Framework Serverless Application Repository - _(SAR)_, it can be found - [here](https://console.aws.amazon.com/lambda/home?region=us-east-1#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:112893979820:applications/aws-deployment-framework). -2. You can find the latest version in the title of the page, like so: - `aws-deployment-framework — version x.y.z`. +[Open Source ADF Repository on GitHub to browse through its +releases](https://github.com/awslabs/aws-deployment-framework/releases). ## Updating Between Versions -Go to the management account in `us-east-1`: - -1. Navigate to the AWS Deployment Framework Serverless Application Repository - _(SAR)_, it can be found - [here](https://console.aws.amazon.com/lambda/home?region=us-east-1#/create/app?applicationId=arn:aws:serverlessrepo:us-east-1:112893979820:applications/aws-deployment-framework). -2. To ease maintaining your ADF installation, it is recommended to ensure that - the values specified reflect what is installed/in use at the moment. - To gather the values, you can either find them in the - `aws-deployment-framework-bootstrap` repository in the `adfconfig.yml` - file. Or by looking up the values that were specified the last time ADF got - installed/updated via the CloudFormation template parameters of the - `serverlessrepo-aws-deployment-framework` stack in `us-east-1`. -3. Tick the box at the bottom that states: _"I acknowledge that this app creates - custom IAM roles and resource policies."_ -4. Click the _Deploy_ button. - -This will take a few minutes to deploy and kick-off your SAR deployment using -CloudFormation. Leave the browser window open until it changes pages. +1. Follow the build and deploy steps, as documented in the [installation guide, + steps 1 to 3](./installation-guide.md). -Your `serverlessrepo-aws-deployment-framework` stack is updating -with new changes that were included in that release of ADF. +The `serverlessrepo-aws-deployment-framework` stack is updated through this +process with new changes that were included in that release of ADF. To check the progress in the management account in `us-east-1`, follow these steps: @@ -980,7 +959,7 @@ steps: in your AWS management account too. To ease this process, the AWS CloudFormation stack will run the - _InitialCommit_ Custom CloudFormation resource when updating via the SAR. + _InitialCommit_ Custom CloudFormation resource when updating ADF. This resource will open a pull request against the default branch (i.e. `main`) on the _bootstrap_ repository with a set of changes that you can optionally choose to merge. If those changes are merged into the default diff --git a/docs/installation-guide.md b/docs/installation-guide.md index 0ae5b640d..4a8c43c68 100644 --- a/docs/installation-guide.md +++ b/docs/installation-guide.md @@ -1,19 +1,22 @@ # Installation Guide -## Pre-Requisites +## Prerequisites - [awscli](https://aws.amazon.com/cli/). - [git](https://git-scm.com/) - [AWS CodeCommit Setup](https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-https-unixes.html) -- [AWS CloudTrail configured](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html) - in the `us-east-1` region within the AWS Organizations Management AWS Account. +- [docker](https://docs.docker.com/get-docker/) -## ADF-Compability with AWS Control Tower +The steps in this guide should be performed in the AWS Organizations Management +account. + +## Compatibility with AWS Control Tower ADF is fully compatible with [AWS Control Tower](https://aws.amazon.com/de/controltower/). ADF augments AWS Control Tower. A common operations model is defined as follows: -- AWS Control Tower is responsible for AWS Account creation and OU mapping. +- AWS Control Tower is responsible for AWS Account creation and Organization + Unit (OU) mapping. - ADF is responsible for deploying applications as defined in the ADF deployment maps. @@ -21,71 +24,598 @@ In the following, we assume that you install ADF without AWS Control Tower. However, if a specific installation step requires a "AWS Control Tower-specific action, we call those out explicitly. -It is okay to install ADF and AWS Control Tower in different regions. Example: +It is okay to install ADF and AWS Control Tower in different regions. +For example: -- Install AWS Control Tower in eu-central-1. -- Install ADF in us-east-1. +- Install AWS Control Tower in `eu-central-1`. +- Install ADF in `us-east-1`. **If you want to use ADF and AWS Control Tower, we recommend that you setup AWS Control Tower prior to installing ADF.** -## Installation Instructions - -1. Ensure you have setup [AWS CloudTrail](https://aws.amazon.com/cloudtrail/) - *(Not the default trail)* in your Management Account that spans **all - regions**, the trail itself can be created in any region. Events [triggered - via - CloudTrail](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_incident-response.html) - for AWS Organizations can only be acted upon in the US East (N. Virginia) - Region. - -2. In the AWS Console from your management account within `us-east-1`, head - over to the Serverless Application Repository *(SAR)*. From there, search - for `aws-deployment-framework` *(or "adf")* (ensure the checkbox - *"Show apps that create custom IAM roles or resource policies"* is checked). - - If you are deploying ADF for the first time, fill in the required parameters - for your specific use-case. For example, if you have no AWS Organization - or dedicated deployment account already created, you can enter an account - name and email address and ADF will create you an AWS Organization, the - deployment OU, along with an AWS Account that will be used to house - deployment pipelines throughout your Organization. - - If you already have an AWS Account you want to use as your deployment - account you can specify its Account ID in the parameter - `DeploymentAccountId` and leave the `DeploymentAccountName` plus - `DeploymentAccountEmail` empty. - - **AWS Control Tower-specific Note:** - If you use AWS Control Tower, we recommend to create the deployment AWS - Account via the account vending feature of AWS Control Tower. - - It is **MANDATORY**, that your designated deployment AWS Account resides in - the OU `deployment` (case-sensitive!). This can't be changed currently. - Otherwise, the ADF deployment will fail! - - Next, specify the `DeploymentAccountMainRegion` parameter as the region that - will host your deployment pipelines and would be considered your main AWS - region. - - In the `DeploymentAccountTargetRegions` section of the parameters - enter a list of AWS Regions that you might want to deploy your resources - or applications into via AWS CodePipeline *(this can be updated whenever)*. - Also specify a main notification endpoint *(email)* to receive updates - about the bootstrap process. - - **AWS Control Tower-specific Note:** - If you use AWS Control Tower, in the `CrossAccountAccessRoleName` section of - the parameters enter the string `AWSControlTowerExecution`. - Alternatively, leave empty for a default ADF setup. - - When you have entered all required information press **'Deploy'**. - -3. As the stack `serverlessrepo-aws-deployment-framework` completes you can now +--------------------------------- + +## 1. Enable CloudTrail + +Ensure you have setup [AWS CloudTrail](https://aws.amazon.com/cloudtrail/) +*(Not the default trail)* in your Management Account that spans **all +regions**, the trail itself can be created in any region. Events [triggered via +CloudTrail](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_incident-response.html) +for AWS Organizations can only be acted upon in the us-east-1 (North Virginia) +region. + +Please use the [AWS CloudTrail +instructions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html) +to configure the CloudTrail in the `us-east-1` region within the AWS +Organizations Management AWS Account. + +## 2. Setup Your Build Environment + +### 2.1. Local Instructions + +For the following steps, we need to work on a Linux/Unix/macOS environment. +Additionally, you need to have AWS credentials configured to perform the +deployment actions in the AWS Organization Management account. + +Please note that building on *Windows* is not supported, please use the +[Cloud9 instructions](#22-cloud9-instructions) below instead. + +- [git](https://git-scm.com/) + - To test if it is available, run `git --version`. + This should return version 2.30 or later. +- [docker](https://docs.docker.com/get-docker/) + - To test if it is available, run: `docker ps -a`. + This should return a table that is possibly empty. + - Additionally, running `docker --version` should return version 19 or + later. +- [make](https://www.gnu.org/software/make/) + - To test if it is available, run `make --version`. + This should return 4.3 or later. +- [python 3](https://www.python.org/downloads/) + - To test if it is available, run `python --version`. + This should return 3.11 or later. + +Install or update the above requirements as needed. +Alternatively, jump to the [Cloud9 instructions](#22-cloud9-instructions) instead +to use a cloud development machine to get started quickly. + +After your environment is setup, please follow the [Build and Deploy +instructions)](#3-build-and-deploy-instructions) to continue. + +### 2.2. Cloud9 Instructions + +These instructions help setup and configure your Cloud9 instance to build +and deploy ADF with in case your local environment did not meet the +requirements. If your local setup is configured and working, feel free to skip +this section and jump to the [Build and Deploy +instructions)](#3-build-and-deploy-instructions) instead. + +Read this [guide](https://docs.aws.amazon.com/cloud9/latest/user-guide/welcome.html) +to learn about AWS Cloud9 and how-to get started. + +Either deploy the Cloud9 in the management account itself, or configure AWS +credentials such that you can perform actions in the AWS Management account. + +Please pick an AWS Cloud9 compatible region to run the instance from. +During the deployment action of ADF, it will ask to perform the deployment +in the AWS Organizations control-plane region. Obviously, the credentials +available or configured in the Cloud9 instance should have access rights to +deploy ADF in the management account in that region. + +The Cloud9 instance is needed to build and deploy ADF. Feel free to delete +the Cloud9 environment after ADF is deployed successfully. + +#### 2.2.1. EC2 Instance + +Any instance type would work. + +The platform should be: **Amazon Linux 2023** or **Ubuntu Server 22.04 LTS** +or later. + +#### 2.2.2. Network Settings + +To build and deploy ADF, you will need to setup a Cloud9 environment that has +credentials setup. The recommended approach as documented +[here](https://docs.aws.amazon.com/cloud9/latest/user-guide/credentials.html) +is to use AWS Managed Temporary Credentials. In Cloud9, it will be able +to operate with your access rights (similarly to how AWS CloudShell works) if +you set it up: + +- With SSM, no-ingress. Please note, use SSM as in AWS Systems Manager, *not* + SSH. +- Leave VPC to the default, or place the machine in a public subnet. + +#### 2.2.3. Free Disk Space + +Once the instance is created, you need at least 16 GB of free disk space. +Unfortunately, the default Cloud9 instance volume is too small. + +To change this, the quickest solution is to open the Cloud9 IDE and follow +the [resize the EBS volume guide in the Cloud9 +documentation](https://docs.aws.amazon.com/cloud9/latest/user-guide/move-environment.html#move-environment-resize). + +At the moment of writing, the volume Cloud9 creates is 10 GB, with around 2 GB +of free space. It is recommended to increase this to: **24 GB or larger**. + +Please confirm the free disk space is large enough by running: + +```bash +df -h $PWD +``` + +## 3. Build and Deploy Instructions + +### 3.1. Clone The Repository + +If this is the first-time you clone the repository in your build environment, +please run: + +```bash +git clone https://github.com/awslabs/aws-deployment-framework +``` + +If you cloned it before, you do not need to clone it again. + +### 3.2. Checkout Specific Version + +Before we continue, make sure you navigate into the cloned repository: + +```bash +cd aws-deployment-framework +``` + +By default, if you clone a repository, it checks out the main branch of the +repository. +It is strongly recommended to checkout the tag of the version you would like to +deploy. This makes it easier to investigate potential issues later on. + +First we need to update our local copy and fetch the available versions: + +```bash +git fetch --verbose --tags +``` + +Select the version you would like to deploy from the returned list of versions. +You can checkout a specific version by running: + +```bash +git checkout ${version_tag_goes_here} + +# For example: +git checkout v3.2.0 +``` + +### 3.3. Update Makefile + +Before we continue, we need to make sure we have the latest build script. + +Update the build script (Makefile) to the latest version by running: + +```bash +make update_makefile +``` + +If this throws an error, the specific version of ADF you are trying to use +might not support this command yet. In that this case, run: + +```bash +RAW_ADF_URL=https://raw.githubusercontent.com/awslabs/aws-deployment-framework +curl -fsSL \ + "${RAW_ADF_URL}/make/latest/Makefile" \ + -o ./Makefile.new +mv ./Makefile.new ./Makefile +``` + +Make sure to run `make clean` next. + +### 3.4. Build ADF + +Build the ADF version with this command next: + +```bash +make +``` + +If this fails, run `make clean` first and try again. + +### 3.5. Notes on a First-Time Deployment + +If you are deploying ADF for the first time, it is recommended to read this +section carefully and determine which parameters are required for your specific +use-case. + +For example, if you have no AWS Organization or dedicated deployment account +yet, you can enter an account name and email address and ADF will create you +an AWS Organization, the deployment organization unit (OU), along with an +AWS Account that will be used to house deployment pipelines throughout your +Organization. + +If you already have an AWS Account you want to use as your deployment +account you can specify its Account ID in the parameter +`DeploymentAccountId` and leave the `DeploymentAccountName` plus +`DeploymentAccountEmail` empty. + +**AWS Control Tower-specific Note:** +If you use AWS Control Tower, we recommend to create the deployment AWS +Account via the account vending feature of AWS Control Tower. + +It is **MANDATORY**, that your designated deployment AWS Account resides in +the `deployment` (case-sensitive!) OU. This cannot be changed currently. +Also make sure there is only one AWS Account in the deployment organization +unit. Without one, or with more than one the ADF deployment will fail! + +The `DeploymentAccountMainRegion` parameter asks for the region that +will host your deployment pipelines and would-be considered your main AWS +region. + +In the `DeploymentAccountTargetRegions` section of the parameters +enter a list of AWS Regions that you might want to deploy your resources +or applications into via AWS CodePipeline *(this can be updated later)*. + +When deploying ADF for the first time, part of the installation process will +automatically create an AWS CodeCommit repository in the management AWS Account +within the `us-east-1` region. It will also make the initial commit to the +default branch of this repository with a default set of examples that act as a +starting point to help define the AWS Account bootstrapping processes for your +Organization. + +Part of the questions that follow will end up in the initial commit into the +repository. These are passed directly the `adfconfig.yml` file prior to it +being committed. + +### 3.6. Notes on Updating ADF + +ADF releases follow the [semantic version number schems](https://semver.org/). +This implies that breaking changes would require a major version upgrade. +Hence if you upgrade from `2.x` to `3.x`, it is recommended to check the +[release notes](https://github.com/awslabs/aws-deployment-framework/releases) +for details and the breaking changes that it introduced. + +### 3.7. Deploy ADF + +Next, to deploy ADF, run: + +```bash +make deploy +``` + +This process will raise several questions, please use the documentation below +to determine what to answer for each. + +Once you deployed ADF this way, it will ask you to save the configuration. +It is recommended to do so and store the `samconfig.toml` file that it +generates in a Wiki or repository. The next time you need to perform an update +you can rely on that configuration file to use the same settings as configured +with the last deployment. + +If this is the first time you deploy using this method, but have an existing +ADF installation. It is recommended to ensure that the values specified reflect +what is installed/in use at the moment. +To gather the values, you can either find them in the +`aws-deployment-framework-bootstrap` repository in the `adfconfig.yml` +file. Or by looking up the values that were specified the last time ADF got +installed/updated via the CloudFormation template parameters of the +`serverlessrepo-aws-deployment-framework` stack in `us-east-1`. + +#### Stack Name + +Recommended value to use: `serverlessrepo-aws-deployment-framework` + +**Explanation:** +This defines the name of the stack with which ADF is deployed +in AWS CloudFormation. In theory, you can customize the stack name. +However, in case of updates of existing installations, you should use the same +name as used before. Otherwise it will attempt to deploy a separate stack aside +of the original one, which will result in a failure. + +If you deployed ADF v3.2.0 or earlier and you are about to upgrade that, the +name that it used to deploy is the recommended value stated above. + +#### AWS Region + +Value to use depends on the AWS partition it is deployed to: + +- For the AWS partition (most common), use; `us-east-1` +- For the US-Gov partition, use: `us-gov-west-1` + +**Explanation:** +ADF needs to be deployed in the region where the control plane of the +AWS Organizations service is hosted. Choose one of the above options based on +the partition you deploy ADF to. + +This does not need to be the same region as you are running the development +environment from with Cloud9 for example. Nor does this have to be the same +region as where all the pipelines will be created by ADF. This later region +is configured at a later stage. + +#### Parameter CrossAccountAccessRoleName + +Default value: `OrganizationAccountAccessRole` + +If your AWS Organization is managed via AWS Control Tower, specify +`AWSControlTowerExecution` instead. You can find more information on [using +AWS Control Tower and ADF here](#compatibility-with-aws-control-tower). + +**Explanation:** +This role is used to deploy the resources for cross-account management by ADF +initially. + +The name of the IAM Role that ADF will use to access other AWS Accounts within +your Organization to create and update base CloudFormation stacks. + +This role must exist in all AWS accounts within your Organization that you +intend to use ADF with. When creating new AWS Accounts via AWS Organizations +you can define an initial role that is created on the account, that role name +should be standardized and can be used as this initial cross-account access +role. + +*This is not required when performing an update between versions of ADF.* + +Please note that changing this value does not change the configured role in an +existing ADF installation. Please update the role to use in the `adfconfig.yml` +file instead, as documented in the [adfconfig section in the Admin +Guide](./admin-guide.md#adfconfig). + +#### Parameter MainNotificationEndpoint + +Optional, default value: (empty) + +Example: `jane@example.com` + +**Explanation:** +This allows you to configure the main notification endpoint that should be +informed in case the ADF bootstrapping pipeline failed on the management +account. + +*This is not required when performing an update between versions of ADF.* + +Please note that changing this value does not change the configured value in an +existing ADF installation. Please update the value to use in the `adfconfig.yml` +file instead, as documented in the [adfconfig section in the Admin +Guide](./admin-guide.md#adfconfig). + +#### Parameter DeploymentAccountName + +Optional, default value: (empty) + +Only required upon first-install, if you want ADF to create a new +deployment account for you. If you specify this, along with the +DeploymentAccountId, it will use the existing deployment account instead +and ignore this setting. + +Example: `deployment` + +**Explanation:** +The Name of the centralized Deployment Account. +If you have an existing account you wish to use as the deployment account, +insert the name of the AWS account here. + +*This is not required when performing an update between versions of ADF.* +*Only supported when installing ADF for the first time.* + +Please note that changing this value does not change the account name in an +existing ADF installation. + +#### Parameter DeploymentAccountEmailAddress + +Optional, default value: (empty) + +Example: `jane@example.com` + +Only required upon first-install, if you want ADF to create a new +deployment account for you. If you specify this, along with the +DeploymentAccountId, it will use the existing deployment account instead +and ignore this setting. + +**Explanation:** +The email address associated with the deployment account, +If you have an existing account you wish to use as the deployment account, +insert the email address of the AWS account here. + +*This is not required when performing an update between versions of ADF.* +*Only supported when installing ADF for the first time.* + +Please note that changing this value does not change the account email in an +existing ADF installation. + +#### Parameter DeploymentAccountAlias + +Optional, default value: (empty) + +Example: `deployment` + +**Explanation:** +The Alias of the deployment account. The account alias is a globally unique +name for an account that enable things such as custom login URLs. Read more +[here](https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html#AboutAccountAlias). + +*This is not required when performing an update between versions of ADF.* +*Only supported when installing ADF for the first time.* + +Please note that changing this value does not change the account email in an +existing ADF installation. + +#### Parameter DeploymentAccountId + +Optional, default value: (empty) + +Example: `123456789012` + +Only required upon updates or a first installation if you want ADF to use an +existing deployment account. + +**Explanation:** +The AWS Account number of the **existing** deployment account, only required +if an existing account should be used. + +A deployment account will be created if this value is omitted. + +Please note that changing this value does not change the account to use in an +existing ADF installation. + +#### Parameter DeploymentAccountMainRegion + +Required, no default value. + +Example: `eu-west-1` + +**Explanation:** +The AWS region that will centrally hold all AWS CodePipeline Pipelines. +Pipeline deployments can still span multiple regions. However, they are still +stored and viewed from a single region perspective. +This region is considered your default ADF AWS Region. + +Please note that changing this value does not change the configured region in +an existing ADF installation. You need to remove and reinstall ADF to change +the main region. + +#### Parameter DeploymentAccountTargetRegions + +Optional, default value: (empty) + +Example: `eu-central-1,us-east-1` + +**Explanation:** +An optional comma-separated list of regions that you may want to deploy +resources *(Applications, CloudFormation etc)* into via AWS CodePipeline. +This can always be updated later via the `adfconfig.yml` file. + +You don't need to include the main region in this list. For example, if you +use the example values for the default region and target regions, it will allow +pipelines to deploy to `eu-west-1`, `eu-central-`, and `us-east-1`. + +*This is not required when performing an update between versions of ADF.* +*Only supported when installing ADF for the first time. + +Please note that changing this value does not change the configured regions in +an existing ADF installation. Please update the target regions to use in the +`adfconfig.yml` file instead, as documented in the [adfconfig section in the +Admin Guide](./admin-guide.md#adfconfig). + +#### Parameter ProtectedOUs + +Optional, default value: (empty) + +Example: `ou-123,ou-234` + +**Explanation:** +An optional comma-separated list of Organization Unit identifiers that you may +want to protect against having bootstrap stacks applied. + +*This is not required when performing an update between versions of ADF.* +*Only supported when installing ADF for the first time. + +Please note that changing this value does not change the configured protected +OUs in an existing ADF installation. +Please update the configuration to use in the `adfconfig.yml` +file instead, as documented in the [adfconfig section in the Admin +Guide](./admin-guide.md#adfconfig). + +#### Parameter LogLevel + +Optional, default value: `INFO` + +Example: `DEBUG` +Valid options are: `DEBUG`, `INFO`, `WARN`, `ERROR`, and `CRITICAL`. + +**Explanation:** +At what Log Level the ADF should operate, default is INFO. + +#### Confirm changes before deploy + +Recommended: `Yes` + +**Explanation:** +It is recommended to answer with `Yes` here. +This allows you to check if the change set that is created does not introduce +any breaking changes before it is executed. + +#### Disable rollback + +Recommended: `No` + +**Explanation:** +It is recommended to answer with `No` here. +Named resources like the AWS Lambda Layer cannot be updated if rollback is +disabled. However, disabling rollback could be useful if you are experiencing +issues and you want to investigate. + +#### Save arguments to configuration file + +Recommended: `Yes` + +**Explanation:** +It is recommended to answer with `Yes` here. +This will save answers for these questions to the `samconfig.toml` +configuration file. +It is recommended to store the `samconfig.toml` file in a Wiki or repository. +The next time you need to perform an update you can rely on that configuration +file to use the same settings as configured with the last deployment. + +#### SAM configuration file + +Recommended and default value: `samconfig.toml` + +**Explanation:** +You are allowed to change this to `samconfig.yml` or `samconfig.yaml`. +Any other name is not recommended, as the deployment script would need to know +where to find the file. If you insist on using a specific file, please rename +it after and before you use the build and deploy steps. + +#### SAM configuration environment + +Default value: `default` + +**Explanation:** +You are allowed to change this, this is especially useful if you deploy ADF to +multiple environments. Keeping the configuration together in a single file. + +**Please note:** This is the last question, once you provided the answer it +will continue to upload the assets and build the CloudFormation change-set. + +## What happens next? + +Once the **make deploy** command succeeds, the base AWS Deployment Framework +[CloudFormation stack](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks) +is created. + +This stack contains a few notable resources such as: + +- an [AWS CodeCommit + Repository](https://console.aws.amazon.com/codesuite/codecommit/repositories/aws-deployment-framework-bootstrap/browse?region=us-east-1) +- along with an associated [AWS CodeBuild + Project](https://console.aws.amazon.com/codesuite/codebuild/projects/aws-deployment-framework-base-templates/history?region=us-east-1) +- and an [AWS CodePipeline +Pipeline](https://console.aws.amazon.com/codesuite/codepipeline/pipelines/aws-deployment-framework-bootstrap-pipeline/view?region=us-east-1) + +The CodeCommit Repository will have its first commit made to it automatically +in the installation process. This is a starting point for using ADF and helps +define the bootstrap process for your AWS ecosystem. + +If you updated ADF, it will create a pull request to the main branch on the +same repository instead. + +Once the commits land in the main branch, CodePipeline will kick off the AWS +CodeBuild Project to sync the content of the repository with Amazon S3 and +bootstrap the AWS Accounts where required. + +It is recommended to go through [the steps how ADF handles AWS Account +provisioning as documented in the admin +guide](./admin-guide.md#account-provisioning). + +If you updated ADF, instead of the following steps, +follow the steps described in the [admin guide on updating +ADF](./admin-guide.md#updating-between-versions). + +For first-time deployments, the process that deploys ADF continues +automatically in the background, to follow its progress: + +1. Please navigate to the AWS Console in the AWS Management account. + As the stack `serverlessrepo-aws-deployment-framework` completes you can now open AWS CodePipeline from within the management account in `us-east-1` and - see that there is an initial pipeline execution that has been started. + see that there is an initial pipeline execution that started. - When ADF is deployed for the first time, it will make the initial commit + When ADF is deployed for the first-time, it will make the initial commit with the skeleton structure of the `aws-deployment-framework-bootstrap` CodeCommit repository. @@ -94,26 +624,27 @@ AWS Control Tower prior to installing ADF.** via AWS CloudFormation Templates, Service Control Policies or Tagging Policies. -4. As part of the AWS CodePipeline Execution from the previous step, the +2. As part of the AWS CodePipeline Execution from the previous step, the account provisioner component will run *(in CodeBuild)*. - OPTION 4.1: ONLY applies when requesting the creation of a NEW deployment + OPTION 2.1: ONLY applies when requesting the creation of a NEW deployment account AND when using ADF for vending AWS Accounts. - - If you let ADF create a new Deployment account for you - *(by not giving a pre-existing account id when deploying from SAR)*, + - If you let ADF create a new deployment account for you + *(by not giving a pre-existing account id for the deployment account + at the time of ADF deployment)*, then ADF will handle creating and moving this account automatically into the deployment OU. - OPTION 4.2: ONLY applies when reusing an pre-created deployment account - AND when using ADF for vending AWS Accounts + OPTION 2.2: ONLY applies when reusing a pre-created deployment account + AND when using ADF for vending AWS Accounts. - - If you are using a pre-existing deployment account, you will need to - move the account into the deployment OU from within the Organization - console, or add your deployment account into a `.yml` file within the - `adf-accounts` folder *(see docs)*. + - If you are using a pre-existing deployment account, ADF will + move the account into the deployment OU automatically. + It will also configure your deployment account into a `.yml` file + within the `adf-accounts` folder. - OPTION 4.3: ONLY applies when reusing a pre-existing deployment account + OPTION 2.3: ONLY applies when reusing a pre-existing deployment account AND when using AWS Control Tower for vending AWS Accounts - Ensure that the AWS Control Tower-created deployment AWS Account @@ -126,9 +657,9 @@ AWS Control Tower prior to installing ADF.** console for the step function `AccountBootstrappingStateMachine-` in the `us-east-1` region. -5. Once the Step Function has completed, switch roles over to the newly +3. Once the Step Function has completed, switch roles over to the newly bootstrapped deployment account in the region you defined as your main - region from step 2. + region at ADF deployment time. An AWS CodeCommit repository will have been created and will contain the initial skeleton structure committed which serves as a starting point for diff --git a/docs/serverless-application-repo.md b/docs/serverless-application-repo.md deleted file mode 100644 index c3a784657..000000000 --- a/docs/serverless-application-repo.md +++ /dev/null @@ -1,135 +0,0 @@ -# AWS Deployment Framework - -[![Build Status](https://github.com/awslabs/aws-deployment-framework/workflows/ADF%20CI/badge.svg?branch=master)](https://github.com/awslabs/aws-deployment-framework/actions?query=workflow%3AADF%20CI+branch%3Amaster) - -The [AWS Deployment Framework](https://github.com/awslabs/aws-deployment-framework) -*(ADF)* is an extensive and flexible framework to manage and deploy resources -across multiple AWS accounts and regions within an AWS Organization. This -application should be deployed via the SAR in the [Management AWS -account](admin-guide.md#management-account) of your AWS Organization within the -`us-east-1` region. For more information on setting up ADF please see the -[installation -guide](https://github.com/awslabs/aws-deployment-framework/tree/master/docs/installation-guide.md). - -- **Application Name:** The stack name of this application created via AWS - CloudFormation. By default, this should be `aws-deployment-framework`. - -- **CrossAccountAccessRoleName:** The name of the IAM Role that ADF will use to - access other AWS Accounts within your Organization to create and update base - CloudFormation stacks. This role must exist in all AWS accounts within your - Organization that you intend to use ADF with. When creating new AWS Accounts - via AWS Organizations you can define an initial role that is created on the - account, that role name should be standardized and can be used as this initial - cross-account access role. - - *This is not required when performing an update between versions of ADF.* - -- **DeploymentAccountEmailAddress:** The Email address associated with the - Deployment Account, If you have an existing account you wish to use as the - deployment account, insert the email address of the AWS account here. - - *This is not required when performing an update between versions of ADF.* - -- **DeploymentAccountId:** The AWS Account number of the **existing** Deployment - Account, only required if an existing account should be used. A deployment - account will be created if this value is omitted. Only required if using - pre-existing AWS Account as the Deployment Account. - - *This is not required when performing an update between versions of ADF.* - -- **DeploymentAccountMainRegion:** The AWS region that will centrally hold all - AWS CodePipeline Pipelines. Pipeline deployments can still span multiple - regions however they are still stored and viewed from a single region - perspective. This would be considered your default ADF AWS Region. - - *This is not required when performing an update between versions of ADF.* - -- **DeploymentAccountName:** The Name of the centralized Deployment Account. If - you have an existing account you wish to use as the deployment account, insert - the name of the AWS account here. *This is not required when performing an - update between versions of ADF.* - -## InitialCommit - -These are parameters that relate to the setting up the `adfconfig.yml` file and -its initial commit to the bootstrap repository. The `adfconfig.yml` file defines -base level settings for how ADF operates. - -When deploying ADF for the first time, part of the installation process will -automatically create an AWS CodeCommit repository on this AWS Account within the -`us-east-1` region. It will also make the initial commit to the default branch -of this repository with a default set of examples that act as a starting point -to help define the AWS Account bootstrapping processes for your Organization. - -When making this initial commit into the repository, these below settings are -passed directly the `adfconfig.yml` file prior to it being committed. - -- **DeploymentAccountAlias:** The Alias of the Deployment Account. The account - alias is a globally unique name for an account that enable things such as - custom login URLs. Read more - [here](https://docs.aws.amazon.com/IAM/latest/UserGuide/console_account-alias.html#AboutAccountAlias) - -- **DeploymentAccountTargetRegions:** An optional comma separated list of - regions that you may want to deploy resources *(Applications, CloudFormation - etc)* into via AWS CodePipeline, this can always be updated later via the - `adfconfig.yml` file. **(e.g. `us-west-1`,`eu-west-1`)**. - - *This is not required when performing an update between versions of ADF.* - -- **MainNotificationEndpoint:** An optional Email Address that will receive - notifications in regards to the bootstrapping pipeline on the management - account. - - *This is not required when performing an update between versions of ADF.* - -- **ProtectedOUs:** An optional comma separated list of OU ids that you may want - to protect against having bootstrap stacks applied **(e.g. - `ou-123`,`ou-234`)**. - - *This is not required when performing an update between versions of ADF.* - -## What happens next? - -After hitting **Deploy** the base AWS Deployment Framework -[CloudFormation stack](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks) -will be created. - -This Stack contains a few notable resources such as: - -- an [AWS CodeCommit - Repository](https://console.aws.amazon.com/codesuite/codecommit/repositories/aws-deployment-framework-bootstrap/browse?region=us-east-1) -- along with an associated [AWS CodeBuild - Project](https://console.aws.amazon.com/codesuite/codebuild/projects/aws-deployment-framework-base-templates/history?region=us-east-1) -- and an [AWS CodePipeline -Pipeline](https://console.aws.amazon.com/codesuite/codepipeline/pipelines/aws-deployment-framework-bootstrap-pipeline/view?region=us-east-1) - -The CodeCommit Repository will have its first commit made to it automatically by -the ADF installation process. This is a starting point for using ADF and helps -define the bootstrap process for your AWS Ecosystem. - -Once the initial commit has been made, CodePipeline will run which kicks off AWS -CodeBuild to sync the content of the repository with Amazon S3. - -Once the content is synced, you are ready to bootstrap your Deployment account -by moving it into the deployment OU within AWS Organizations. - -Before bootstrapping an AWS Account its important to understand how ADF handles -AWS Account provisioning. Read more about [AWS Account -Provisioning](./admin-guide.md#account-provisioning) in the admin guide. - -## Upgrading between versions? - -Ensure the **CrossAccountAccessRole** and **Application Name** are the same -value you used for your initial deployment. Click **Deploy**. As part of the -installation phase, a custom CloudFormation resource will make a Pull Request -against the [AWS CodeCommit -Repository](https://console.aws.amazon.com/codesuite/codecommit/repositories/aws-deployment-framework-bootstrap/browse?region=us-east-1) -with the content from the latest release of ADF. - -Ensure none of your specific bootstrap templates are being overwritten or -affected in any way before merging. Once merged, the pipeline will run, this -completes the update process. - -Upgrading from `*2.x` to `3.x*`? See the -[3.0 release notes](https://github.com/awslabs/aws-deployment-framework/releases) -for details. diff --git a/requirements-dev.txt b/requirements-dev.txt index af21021c8..0155bdfe2 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -6,3 +6,4 @@ pytest~=7.4.4 pytest-cov==4.1.0 tox==4.11.4 yamllint==1.33.0 +yq==3.2.3 diff --git a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/initial_commit/initial_commit.py b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/initial_commit/initial_commit.py index 4ceaa6adb..a1db0e5ab 100644 --- a/src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/initial_commit/initial_commit.py +++ b/src/lambda_codebase/initial_commit/bootstrap_repository/adf-bootstrap/deployment/lambda_codebase/initial_commit/initial_commit.py @@ -40,7 +40,7 @@ https://github.com/awslabs/aws-deployment-framework/releases/tag/v{0} This PR was automatically created when you deployed version {0} of the -AWS Deployment Framework through the Serverless Application Repository. +AWS Deployment Framework. Review this PR to understand what changes will be made to your bootstrapping repository. If you also made changes to the repository yourself, diff --git a/src/lambda_codebase/initial_commit/initial_commit.py b/src/lambda_codebase/initial_commit/initial_commit.py index 994a890bb..3993c4ad0 100644 --- a/src/lambda_codebase/initial_commit/initial_commit.py +++ b/src/lambda_codebase/initial_commit/initial_commit.py @@ -55,7 +55,7 @@ https://github.com/awslabs/aws-deployment-framework/releases/tag/v{0} This PR was automatically created when you deployed version {0} of the -AWS Deployment Framework through the Serverless Application Repository. +AWS Deployment Framework. Review this PR to understand what changes will be made to your bootstrapping repository. If you also made changes to the repository yourself, diff --git a/src/template.yml b/src/template.yml index b372f9626..ca9ab75dd 100644 --- a/src/template.yml +++ b/src/template.yml @@ -14,7 +14,7 @@ Metadata: Author: AWS ADF Builders Team SpdxLicenseId: Apache-2.0 LicenseUrl: ../LICENSE.txt - ReadmeUrl: ../docs/serverless-application-repo.md + ReadmeUrl: ../docs/installation-guide.md Labels: ["adf", "aws-deployment-framework", "multi-account", "cicd", "devops"] HomePageUrl: https://github.com/awslabs/aws-deployment-framework diff --git a/tox.ini b/tox.ini index 7340d7866..b467c5737 100644 --- a/tox.ini +++ b/tox.ini @@ -49,5 +49,4 @@ deps = -r{toxinidir}/src/lambda_codebase/requirements.txt commands = - make test - make lint + make -f Makefile.tox