Skip to content

Latest commit

 

History

History
253 lines (171 loc) · 7.26 KB

installation-guide-azure.md

File metadata and controls

253 lines (171 loc) · 7.26 KB
Raw

Installation steps

Prerequisites

Common:

  • az cli installed (See azure docs)
  • az login executed

If you have more than one subscription connected to your user, you have to login to the right one using:

az login --tenant <directory group URL, i.e. accountgiantswarm.onmicrosoft.com>

Based on the subscrtiption you want to work on, you might need to adjust the cloud name setting. Run the following command to list subscriptions:

az account list -o table

Check the CloudName column on the subscription you want to work with and run the following command accordingly:

az cloud set --name <CloudName got from command above, i.e. AzureGermanCloud>

Multi-master

By default terraform will create multi-master cluster with 3 master nodes, single master mode can be enabled by setting terraform variable master_count=1 or export env variable export TF_VAR_master_count=1.

Create storage account for terraform state

export NAME="cluster1"
export REGION="westeurope"
az group create -n ${NAME}-terraform -l ${REGION}

az storage account create \
  -n ${NAME}terraform \
  -g ${NAME}-terraform \
  --kind BlobStorage \
  --location ${REGION} \
  --sku Standard_RAGRS \
  --access-tier Cool \
  --https-only true

az storage container create \
  -n ${NAME}-state \
  --public-access off \
  --account-name ${NAME}terraform

az storage container create \
  -n ${NAME}-build \
  --public-access off \
  --account-name ${NAME}terraform

Get access key it will be needed in the next step.

az storage account keys list -g ${NAME}-terraform  --account-name ${NAME}terraform

Enable versioning: go to the azure portal, navigate to the resource group ${NAME}-terraform, then to the storage account ${NAME}terraform. In the left menu, click on Data protection, select the Turn on versioning checkbox and click Save.

Prepare terraform environment

cp -r examples/azure/* ./platforms/azure/giantnetes/
cd ./platforms/azure/giantnetes/

Edit bootstrap.sh. DO NOT PUT passwords and keys into bootstrap.sh as it will be stored as plain text.

Now update the terraform-secrets.yaml file with the azure credentials.

Set the storage account access key under the Terraform.ArmAccessKey key.

opsctl update secret --in=terraform-secrets.yaml -k Terraform.ArmAccessKey

If you need to setup a VPN (mandatory for production installations) you first need to get a /28 subnet unique for this installation.

Go to https://github.com/giantswarm/giantswarm/wiki/Giant-Swarm-VPN, choose an unused subnet and add it to the page with the new installation name to reserve it.

Then, set the following variables in the bootstrap.sh file:

export TF_VAR_vpn_enabled=1
export TF_VAR_vpn_right_gateway_address_0=<ip address of first IPSec server (copy this from other installations in the installations repo)>
export TF_VAR_vpn_right_gateway_address_1=<ip address of second IPSec server (copy this from other installations in the installations repo)>
export TF_VAR_bastion_cidr=<the subnet you have chosen>

When you have finished editing the file, source it:

source bootstrap.sh

NOTE: Reexecute source bootstrap.sh everytime if opening new console.

Install

Terraform has one manifest:

  • platforms/azure/giantnetes - all cluster resources

Install consists two stages:

  • Vault (only needed because we bootstrapping Vault manually)
  • Kubernetes

Stage: Vault

Create Vault virtual machine and all other necessary resources

source bootstrap.sh

terraform plan ./
terraform apply -target="module.dns" ./ 
terraform apply -target="module.vnet" -target="module.bastion" -target="module.vault" -target="module.vpn" ./

(Optional) Connect to VPN

If VPN was enabled, two additional manual steps are required:

  1. Create VPN connection clients on Azure with a shared key you generate randomly.
export SHARED_KEY=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 50 | head -n 1)

echo $SHARED_KEY

az network vpn-connection create \
  -g ${NAME} \
  --name ${NAME}-vpn-connection-0 \
  --vnet-gateway1 ${NAME}-vpn-gateway \
  --local-gateway2 ${NAME}-vpn-right-gateway-0 \
  --shared-key $SHARED_KEY

az network vpn-connection create \
  -g ${NAME} \
  --name ${NAME}-vpn-connection-1 \
  --vnet-gateway1 ${NAME}-vpn-gateway \
  --local-gateway2 ${NAME}-vpn-right-gateway-1 \
  --shared-key $SHARED_KEY
  1. Update the VPN connection

Temporarily save the password generated somewhere, then follow the instructions at the following page to update the VPN servers: https://github.com/giantswarm/vpn#configure-new-site2site-vpn-with-aws-installation

Provision Vault with Ansible

How to do that see here

Stage: Kubernetes

# Need to source the bootstrap.sh file again to read the new secret defined above.
source bootstrap.sh
terraform apply ./

Complete Vault setup

Setup the Vault Kubernetes Auth backend by following this guide.

Upload variables and configuration

Create terraform folder in installations repository under particular installation folder. Copy variables and configuration.

export INSTALLATIONS=<installations_repo_path>

mkdir -p ${INSTALLATIONS}/${NAME}/terraform
cp bootstrap.sh terraform-secrets.yaml ${INSTALLATIONS}/${NAME}/terraform/

cd ${INSTALLATIONS}
git checkout -b "${NAME}_terraform"
git add ${INSTALLATIONS}/${NAME}/terraform
git commit -S -m "Add ${NAME} terraform variables and configuration"

Create PR with related changes.

Deletion

Easiest way to delete whole cluster is to delete resource group.

az group delete -n <cluster name>

Delete service principal.

az ad sp list --output=table | grep <cluster name> | awk '{print $1}'
az ad sp delete --id <appid>

Updating cluster

Prepare variables and configuration.

cd ./platforms/azure/giantnetes/
export NAME=cluster1
export INSTALLATIONS=<installations_repo_path>

cp ${INSTALLATIONS}/${CLUSTER}/terraform/* .

source bootstrap.sh

Apply latest state

Check resources that has been changed.

terraform plan ./

Update masters

terraform taint module.master.azurerm_virtual_machine.master[0]
terraform apply -target=module.master ./

terraform taint module.master.azurerm_virtual_machine.master[1]
terraform apply -target=module.master ./

terraform taint module.master.azurerm_virtual_machine.master[2]
terraform apply -target=module.master ./

Update everything else

terraform apply ./

NB: worker nodes will be rolled automatically and sequentially by Azure.

Known issues