add e2e test

Bump minimist from 1.2.5 to 1.2.6 in /portal Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> cleanup: proxy now uses idiomatic waitgroup. cleanup: removed useless anonymous function definition. add containers_image_openpgp tag (Azure#2032) Change secrets-update to allow subsequent updates (Azure#2038) Co-authored-by: Nont <[email protected]> add containers_image_openpgp everywhere add controller into operator for machine health check (Azure#1950) * add worker only controller with operator for machine health check * align mhc node selector pattern with osd Create 2022-04-01 API (Azure#1876) check for default ingressIP when ingressProfiles > 1 (Azure#2021) Signed-off-by: Karan.Magdani <[email protected]> Skip Linux AZ Sec Pack policies from running on VMSS creation (Azure#2041) Admin Portal v2 (Azure#2019) Add in sre portal v2, still default to v1 Co-authored-by: Amber Brown <[email protected]> Co-authored-by: Brett Embery <[email protected]> Co-authored-by: Ben Vesel <[email protected]> Bump minimist from 1.2.5 to 1.2.6 in /portal/v2 (Azure#2043) Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6. - [Release notes](https://github.com/substack/minimist/releases) - [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6) --- updated-dependencies: - dependency-name: minimist dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> added changes to make local e2e test work/ update doc (Azure#2036) * added changes to make local e2e test work/ update doc updated operator README to include instructions for running the ARO operator locally for a private cluster (Azure#2045) Fix off by one error when truncating name Now it truncates to 14 instead of 15. the corresponding arm templates truncate to 15. Refactors createOrUpdateRouterIPFromCluster Make it reuse isIngressProfileAvailable to check IngressProfile Adds an extra case into TestAroDeploymentReady Updates dev env docs * Removes mention of Python virtualenv as it comes by default with Python 3 * Updates macOS docs to make sure that steps work for Intel and ARM macs * Markdown formatting fixes give /tmp a bit more room for when the CI VM gets busy refactor+test: refactored some functions to test refactored tests added license to test file added err check on validateProxyResquest made the errors more explicit fixed typo in function name removed useless test case renamed oddly named metrics.Interface to Emitter update codeowners renamed github username updated path to quota file (Azure#2058) refactor/add-test : refactored linkid and gateway to add tests (Azure#2013) Enable first basic linters in ARO (Azure#2060) * Enable first basic linters in ARO * Remove modules-download-mode from the linter run config Commit to allow password auth for VMSS jit access (Azure#2027) * Commit to allow password auth for VMSS jit access fix: now uses renamed interface metricsEmitter fix issues with linting new test files added doc.go for imgconfig controller (Azure#2064) Signed-off-by: Karan.Magdani <[email protected]> Revert 2027: Commit to allow password auth for VMSS jit access Add logic to reconcile failed Nic on az aro delete Co-authored-by: Ben Vesel <[email protected]> Update pull secret references from cloud.redhat.com to cloud.openshift.com (Azure#2084) Enables go fmt simplify (Azure#2081) update reference to cloud.redhat.com in README file (Azure#2085) ensure apiserverready check redesigned the quota computation to something understandable (Azure#2059) Bump 4.9 install image to latest stable 4.9.28 to address etcd split brain issue Fail MUO test if we expect an error but don't get one Bump fluentbit, mdm, and mdsd images to mitigate P0/P1s Bump async from 2.6.3 to 2.6.4 in /portal/v2 Bumps [async](https://github.com/caolan/async) from 2.6.3 to 2.6.4. - [Release notes](https://github.com/caolan/async/releases) - [Changelog](https://github.com/caolan/async/blob/v2.6.4/CHANGELOG.md) - [Commits](caolan/async@v2.6.3...v2.6.4) --- updated-dependencies: - dependency-name: async dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Update the secret rotate time to 7 days during RP deploy (Azure#2051) Remove dead mirror code referencing 4.3 version which isn't mirrored (Azure#2092) add MTU to the internal OCP Document make generate before mock added unit tests for two new functions fix import order remove trailing spaces make validate-go wants to add trailing lines again found/fixed trailing new line add new line at end of test file added admin update method to adminupdate tests newlinw fixed unit test issue add helper method Improve comment gofmt Remove ACR Image Override (Azure#2090) added stylecheck and moved golangci-lint to a github action (Azure#2083) * enabled github action instead of running from ADO * fixed style * fixed some style fixed styling fixed failing tests because of case on errs Small updates to shared rp docs (Azure#2079) "note" syntax adjustments Small updates to shared rp docs from working sessions added note related to gwy keyvault not being in dev Update docs/prepare-a-shared-rp-development-environment.md Language adjustment. Committing syntax change per Caden's suggestion. Co-Authored-By: Caden Marchese <[email protected]> Co-authored-by: Caden Marchese <[email protected]> Additional gateway tests (Azure#2062) * Add coverage for pkg/gateway. Gateway creation now fails fast when env properties are missing. * refactor large test into multiple test cases Move gateway fluentbit to container Bump async from 2.6.3 to 2.6.4 in /portal/v1 Bumps [async](https://github.com/caolan/async) from 2.6.3 to 2.6.4. - [Release notes](https://github.com/caolan/async/releases) - [Changelog](https://github.com/caolan/async/blob/v2.6.4/CHANGELOG.md) - [Commits](caolan/async@v2.6.3...v2.6.4) --- updated-dependencies: - dependency-name: async dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> set MDSD_MSGPACK_SORT_COLUMNS to perf column sorting on MDSD side and try to avoid hitting max schema count (Azure#2095) Remove mwoodson from codeowners (Azure#2106) Updated FIPs e2e test for 2022-04-01 API Development subscription migration prepare for dns migration Signed-off-by: Karan.Magdani <[email protected]>
ellis-johnson · Jul 22, 2022 · 1ab03e1 · 1ab03e1
1 parent 5983ef8
commit 1ab03e1
Show file tree

Hide file tree

Showing 71 changed files with 1,264 additions and 7,010 deletions.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -1 +1 @@
-* @jewzaam @m1kola @bennerv @hawkowl @mwoodson @rogbas @petrkotas @bryanro92
+* @jewzaam @m1kola @bennerv @hawkowl @rogbas @petrkotas @ross-bryan
diff --git a/.github/workflows/golint.yml b/.github/workflows/golint.yml
@@ -1,4 +1,4 @@
-name: golang-lint
+name: golangci-lint
 on:
   push:
     tags:
@@ -13,11 +13,11 @@ permissions:
   # pull-requests: read
 jobs:
   golangci:
-    name: golangci-lint
+    name: lint
     runs-on: ubuntu-latest
     steps:
-      - run: |
-          sudo apt-get update
+      - run: | 
+          sudo apt-get update 
           sudo apt-get install libgpgme-dev libgpgme11
       - uses: actions/setup-go@v3
         with:
@@ -28,7 +28,7 @@ jobs:
         with:
           # Optional: version of golangci-lint to use in form of v1.2 or v1.2.3 or `latest` to use the latest version
           version: v1.45.2
-          args: -v --timeout 15m
+
           # Optional: working directory, useful for monorepos
           #working-directory: pkg
 
@@ -47,12 +47,3 @@ jobs:
 
           # Optional: if set to true then the action don't cache or restore ~/.cache/go-build.
           # skip-build-cache: true
-  validate-go:
-    name: validate-go
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/setup-go@v3
-        with:
-          go-version: 1.17
-      - uses: actions/checkout@v3
-      - run: make validate-go-action
diff --git a/.golangci.yml b/.golangci.yml
@@ -1,10 +1,7 @@
 run:
-  timeout: 10m
+  timeout: 5m
   skip-dirs:
     - vendor/portal
-    - vendor
-  skip-dirs-use-default: true
-  modules-download-mode: vendor
 
 issues:
   exclude-rules:
@@ -24,6 +21,7 @@ linters-settings:
       - github.com/onsi/ginkgo
       - github.com/onsi/gomega
 
+
 linters:
   disable-all: true
   enable:

diff --git a/.pipelines/ci.yml b/.pipelines/ci.yml
@@ -22,7 +22,6 @@ variables:
 - template: vars.yml
 
 jobs:
-
 - job: Python_Unit_Tests
   pool:
     name: ARO-CI
@@ -53,12 +52,6 @@ jobs:
       [[ -z "$(git status -s)" ]]
     displayName: 🕵️ Validate Golang code
 
-  - script: |
-      set -xe
-      make lint-go
-      [[ -z "$(git status -s)" ]]
-    displayName: 🕵️ Lint Golang code
-
   - script: |
       set -xe
       make build-all
@@ -95,3 +88,11 @@ jobs:
       failIfCoverageEmpty: false
     condition: succeededOrFailed()
 
+- job: Lint_Admin_Portal
+  pool:
+    name: ARO-CI
+  steps:
+  - script: |
+      set -xe
+      make lint-admin-portal
+    displayName: 🧹 Lint Admin Portal
diff --git a/Dockerfile.portal_lint b/Dockerfile.portal_lint
@@ -1,5 +1,4 @@
-ARG REGISTRY
-FROM ${REGISTRY}/ubi8/nodejs-14
+FROM registry.access.redhat.com/ubi8/nodejs-14
 WORKDIR /data
 USER root
 

diff --git a/Makefile b/Makefile
@@ -5,7 +5,7 @@ ARO_IMAGE_BASE = ${RP_IMAGE_ACR}.azurecr.io/aro
 E2E_FLAGS ?= -test.timeout 180m -test.v -ginkgo.v -ginkgo.noColor
 
 # fluentbit version must also be updated in RP code, see pkg/util/version/const.go
-FLUENTBIT_VERSION = 1.7.8-1
+FLUENTBIT_VERSION = 1.9.1-1
 FLUENTBIT_IMAGE ?= ${RP_IMAGE_ACR}.azurecr.io/fluentbit:$(FLUENTBIT_VERSION)
 AUTOREST_VERSION = 3.3.2
 AUTOREST_IMAGE = "quay.io/openshift-on-azure/autorest:${AUTOREST_VERSION}"
@@ -29,7 +29,7 @@ aro: generate
 	go build -tags aro,containers_image_openpgp,codec.safe -ldflags "-X github.com/Azure/ARO-RP/pkg/util/version.GitCommit=$(VERSION)" ./cmd/aro
 
 runlocal-rp:
-	go run -tags aro -ldflags "-X github.com/Azure/ARO-RP/pkg/util/version.GitCommit=$(VERSION)" ./cmd/aro rp
+	go run -tags aro,containers_image_openpgp -ldflags "-X github.com/Azure/ARO-RP/pkg/util/version.GitCommit=$(VERSION)" ./cmd/aro rp
 
 az: pyenv
 	. pyenv/bin/activate && \
@@ -45,12 +45,12 @@ clean:
 	find -type d -name 'gomock_reflect_[0-9]*' -exec rm -rf {} \+ 2>/dev/null
 
 client: generate
-	hack/build-client.sh "${AUTOREST_IMAGE}" 2020-04-30 2021-09-01-preview
+	hack/build-client.sh "${AUTOREST_IMAGE}" 2020-04-30 2021-09-01-preview 2022-04-01
 
 # TODO: hard coding dev-config.yaml is clunky; it is also probably convenient to
 # override COMMIT.
 deploy:
-	go run -tags aro -ldflags "-X github.com/Azure/ARO-RP/pkg/util/version.GitCommit=$(VERSION)" ./cmd/aro deploy dev-config.yaml ${LOCATION}
+	go run -tags aro,containers_image_openpgp -ldflags "-X github.com/Azure/ARO-RP/pkg/util/version.GitCommit=$(VERSION)" ./cmd/aro deploy dev-config.yaml ${LOCATION}
 
 dev-config.yaml:
 	go run ./hack/gendevconfig >dev-config.yaml
@@ -109,10 +109,11 @@ proxy:
 	go build -ldflags "-X github.com/Azure/ARO-RP/pkg/util/version.GitCommit=$(VERSION)" ./hack/proxy
 
 run-portal:
-	go run -tags aro -ldflags "-X github.com/Azure/ARO-RP/pkg/util/version.GitCommit=$(VERSION)" ./cmd/aro portal
+	go run -tags aro,containers_image_openpgp -ldflags "-X github.com/Azure/ARO-RP/pkg/util/version.GitCommit=$(VERSION)" ./cmd/aro portal
 
 build-portal:
-	cd portal && npm install && npm run build
+	cd portal/v1 && npm install && npm run build && cd ../v2 && npm install && npm run build
+	make generate
 
 pyenv:
 	python3 -m venv pyenv
@@ -132,7 +133,7 @@ secrets:
 secrets-update:
 	@[ "${SECRET_SA_ACCOUNT_NAME}" ] || ( echo ">> SECRET_SA_ACCOUNT_NAME is not set"; exit 1 )
 	tar -czf secrets.tar.gz secrets
-	az storage blob upload -n secrets.tar.gz -c secrets -f secrets.tar.gz --account-name ${SECRET_SA_ACCOUNT_NAME} >/dev/null
+	az storage blob upload -n secrets.tar.gz -c secrets -f secrets.tar.gz --overwrite --account-name ${SECRET_SA_ACCOUNT_NAME} >/dev/null
 	rm secrets.tar.gz
 
 tunnel:
@@ -161,11 +162,15 @@ validate-fips:
 	hack/fips/validate-fips.sh
 
 unit-test-go:
-	go run ./vendor/gotest.tools/gotestsum/main.go --format pkgname --junitfile report.xml -- -tags=aro -coverprofile=cover.out ./...
+	go run ./vendor/gotest.tools/gotestsum/main.go --format pkgname --junitfile report.xml -- -tags=aro,containers_image_openpgp -coverprofile=cover.out ./...
 
 lint-go:
 	go run ./vendor/github.com/golangci/golangci-lint/cmd/golangci-lint run
 
+lint-admin-portal:
+	docker build -f Dockerfile.portal_lint . -t linter
+	docker run -it --rm localhost/linter ./src --ext .ts
+
 test-python: pyenv az
 	. pyenv/bin/activate && \
 		azdev linter && \

diff --git a/README.md b/README.md
@@ -116,7 +116,11 @@ questions or comments.
 
     * machineset: Ensures that a minimum of two worker replicas are met.
 
-    * machinehealthcheck: Ensures the MachineHealthCheck resource is running as configured. See [machinehealthcheck/doc.go](pkg/operator/controllers/machinehealthcheck/doc.go)
+    * machinehealthcheck: Ensures the MachineHealthCheck resource is running as configured so that at most one worker node at a time is automatically
+      reconciled when not ready for at least 5 minutes.
+        * The CR will only be applied when both `aro.machinehealthcheck.managed` and `aro.machinehealthcheck.enabled` are set to `"true"`.
+        * When `aro.machinehealthcheck.enabled` is `"false"` and `aro.machinehealthcheck.managed` is `"false"` the CR will be removed from the cluster.
+        * If `aro.machinehealthcheck.enabled` is `"false"` no actions will be taken to modify the CR.
         * More information around the MHC CR can be found [in openshift documentation of MHC](https://docs.openshift.com/container-platform/4.9/machine_management/deploying-machine-health-checks.html)
 
     * monitoring: Ensures that the OpenShift monitoring configuration in the `openshift-monitoring` namespace is consistent and immutable.

diff --git a/cmd/aro/operator.go b/cmd/aro/operator.go
@@ -31,6 +31,7 @@ import (
 	"github.com/Azure/ARO-RP/pkg/operator/controllers/genevalogging"
 	"github.com/Azure/ARO-RP/pkg/operator/controllers/imageconfig"
 	"github.com/Azure/ARO-RP/pkg/operator/controllers/machine"
+	"github.com/Azure/ARO-RP/pkg/operator/controllers/machinehealthcheck"
 	"github.com/Azure/ARO-RP/pkg/operator/controllers/machineset"
 	"github.com/Azure/ARO-RP/pkg/operator/controllers/monitoring"
 	"github.com/Azure/ARO-RP/pkg/operator/controllers/muo"
@@ -216,6 +217,10 @@ func operator(ctx context.Context, log *logrus.Entry) error {
 			mgr)).SetupWithManager(mgr); err != nil {
 			return fmt.Errorf("unable to create controller %s: %v", autosizednodes.ControllerName, err)
 		}
+		if err = (machinehealthcheck.NewReconciler(
+			arocli, dh)).SetupWithManager(mgr); err != nil {
+			return fmt.Errorf("unable to create controller %s: %v", machinehealthcheck.ControllerName, err)
+		}
 	}
 
 	if err = (checker.NewReconciler(

diff --git a/cmd/aro/rp.go b/cmd/aro/rp.go
@@ -70,7 +70,7 @@ func rp(ctx context.Context, log, audit *logrus.Entry) error {
 		return err
 	}
 
-	m := statsd.New(ctx, log.WithField("component", "metrics"), _env, os.Getenv("MDM_ACCOUNT"), os.Getenv("MDM_NAMESPACE"), os.Getenv("MDM_STATSD_SOCKET"))
+	m := statsd.New(ctx, log.WithField("component", "metrics"), _env, os.Getenv("MDM_ACCOUNT"), os.Getenv("MDM_NAMESPACE"))
 
 	g, err := golang.NewMetrics(log.WithField("component", "metrics"), m)
 	if err != nil {

diff --git a/docs/admin-portal.md b/docs/admin-portal.md
@@ -21,8 +21,6 @@ You will require Node.js and `npm`. These instructions were tested with the vers
 
 1. Run `make build-portal` from the main directory. This will install the dependencies and kick off the Webpack build, placing the results in `portal/v2/build/`.
 
-  > __NOTE:__ Due to security compliance requirements, the `make build-portal` target pulls from `arointsvc.azurecr.io`. You can either authenticate to this registry using `az acr login --name arointsvc` to pull the image, or modify the $RP_IMAGE_ACR environment variable to point the builds at `registry.access.redhat.com` instead.
-        
 1. Run `make generate`. This will regenerate the golang file containing the portal content to be served.
 
 1. Commit the results of `build-portal` and `generate`.

diff --git a/docs/prepare-a-shared-rp-development-environment.md b/docs/prepare-a-shared-rp-development-environment.md
@@ -27,7 +27,7 @@ locations.
    Set SECRET_SA_ACCOUNT_NAME to the name of the storage account:
 
    ```bash
-   SECRET_SA_ACCOUNT_NAME=rharosecrets
+   SECRET_SA_ACCOUNT_NAME=rharosecretsdev
    ```
 
 1. You will need an AAD object (this could be your AAD user, or an AAD group of
@@ -88,9 +88,9 @@ locations.
    ```
 
    ```bash
+   > __NOTE:__: for macos change the -w0 option for base64 to -b0
    AZURE_ARM_CLIENT_ID="$(az ad app create \
      --display-name aro-v4-arm-shared \
-     --identifier-uris "https://$(uuidgen)/" \
      --query appId \
      -o tsv)"
    az ad app credential reset \
@@ -117,9 +117,9 @@ locations.
    Now create the application:
 
    ```bash
+   > __NOTE:__: for macos change the -w0 option for base64 to -b0
    AZURE_FP_CLIENT_ID="$(az ad app create \
      --display-name aro-v4-fp-shared \
-     --identifier-uris "https://$(uuidgen)/" \
      --query appId \
      -o tsv)"
    az ad app credential reset \
@@ -141,7 +141,6 @@ locations.
    AZURE_RP_CLIENT_ID="$(az ad app create \
      --display-name aro-v4-rp-shared \
      --end-date '2299-12-31T11:59:59+00:00' \
-     --identifier-uris "https://$(uuidgen)/" \
      --key-type password \
      --password "$AZURE_RP_CLIENT_SECRET" \
      --query appId \
@@ -162,7 +161,6 @@ locations.
    AZURE_GATEWAY_CLIENT_ID="$(az ad app create \
      --display-name aro-v4-gateway-shared \
      --end-date '2299-12-31T11:59:59+00:00' \
-     --identifier-uris "https://$(uuidgen)/" \
      --key-type password \
      --password "$AZURE_GATEWAY_CLIENT_SECRET" \
      --query appId \
@@ -177,7 +175,6 @@ locations.
    AZURE_CLIENT_ID="$(az ad app create \
      --display-name aro-v4-tooling-shared \
      --end-date '2299-12-31T11:59:59+00:00' \
-     --identifier-uris "https://$(uuidgen)/" \
      --key-type password \
      --password "$AZURE_CLIENT_SECRET" \
      --query appId \
@@ -194,26 +191,26 @@ locations.
 
    * Go into the Azure Portal
    * Go to Azure Active Directory
-   * Navigate to the `aro-v4-tooling-shared` app page
+   * Navigate to the `aro-v4-tooling-shared` app registration page
    * Click 'API permissions' in the left side pane
-   * Click 'Microsoft Graph'
    * Click 'Add a permission'.
+   * Click 'Microsoft Graph'
    * Select 'Application permissions'
    * Search for 'Application' and select `Application.ReadWrite.OwnedBy`
    * Click 'Add permissions'
    * This request will need to be approved by a tenant administrator. If you are one, you can click the `Grant admin consent for <name>` button to the right of the `Add a permission` button on the app page
 
-1. Set up the RP role definitions and subscription role assignments in your
-   Azure subscription. This mimics the RBAC that ARM sets up.  With at least
-   `User Access Administrator` permissions on your subscription, do:
+1. Set up the RP role definitions and subscription role assignments in your Azure subscription. The usage of "uuidgen" for fpRoleDefinitionId is simply there to keep from interfering with any linked resources and to create the role net new. This mimics the RBAC that ARM sets up. With at least `User Access Administrator` permissions on your subscription, do:
 
    ```bash
+   LOCATION=<YOUR-REGION>
    az deployment sub create \
      -l $LOCATION \
      --template-file deploy/rbac-development.json \
      --parameters \
        "armServicePrincipalId=$(az ad sp list --filter "appId eq '$AZURE_ARM_CLIENT_ID'" --query '[].objectId' -o tsv)" \
        "fpServicePrincipalId=$(az ad sp list --filter "appId eq '$AZURE_FP_CLIENT_ID'" --query '[].objectId' -o tsv)" \
+       "fpRoleDefinitionId"="$(uuidgen)" \
        "devServicePrincipalId=$(az ad sp list --filter "appId eq '$AZURE_CLIENT_ID'" --query '[].objectId' -o tsv)" \
      >/dev/null
    ```
@@ -230,9 +227,9 @@ locations.
    ```
 
    ```bash
+   > __NOTE:__: for macos change the -w0 option for base64 to -b0
    AZURE_PORTAL_CLIENT_ID="$(az ad app create \
      --display-name aro-v4-portal-shared \
-     --identifier-uris "https://$(uuidgen)/" \
      --reply-urls "https://localhost:8444/callback" \
      --query appId \
      -o tsv)"
@@ -241,8 +238,6 @@ locations.
      --cert "$(base64 -w0 <secrets/portal-client.crt)" >/dev/null
    ```
 
-   TODO: more steps are needed to configure aro-v4-portal-shared.
-
 1. Create an AAD application which will fake up the dbtoken client.
 
    1. Create the application and set `requestedAccessTokenVersion`
@@ -255,6 +250,7 @@ locations.
 
    OBJ_ID="$(az ad app show --id $AZURE_DBTOKEN_CLIENT_ID --query objectId)"
 
+   > __NOTE:__: the graph API requires this to be done from a managed machine
    az rest --method PATCH \
       --uri https://graph.microsoft.com/v1.0/applications/$OBJ_ID/ \
       --body '{"api":{"requestedAccessTokenVersion": 2}}'
@@ -372,13 +368,13 @@ az ad app credential reset \
 
 5. The RP makes API calls to kubernetes cluster via a proxy VMSS agent. For the agent to get the updated certificates, this vm needs to be redeployed. Proxy VM is currently deployed by the `deploy_env_dev` function in `deploy-shared-env.sh`. It makes use of `env-development.json`
 
-6. Run `[rharosecrets|aroe2esecrets] make secrets-update` to upload it to your
+6. Run `[rharosecretsdev|aroe2esecrets] make secrets-update` to upload it to your
 storage account so other people on your team can access it via `make secrets`
 
 # Environment file
 
 1. Choose the resource group prefix.  The resource group location will be
-   appended to the prefix to make the resource group name.
+   The resource group location will be appended to the prefix to make the resource group name. If a v4-prefixed environment exists in the subscription already, use a unique prefix.
 
    ```bash
    RESOURCEGROUP_PREFIX=v4
@@ -480,7 +476,7 @@ each of the bash functions below.
    import_certs_secrets
    ```
 
-   Note: in production, three additional keys/certificates (rp-mdm, rp-mdsd, and
+   > __NOTE:__: in production, three additional keys/certificates (rp-mdm, rp-mdsd, and
    cluster-mdsd) are also required in the $KEYVAULT_PREFIX-svc key vault.  These
    are client certificates for RP metric and log forwarding (respectively) to
    Geneva.
@@ -512,10 +508,12 @@ each of the bash functions below.
         --file secrets/cluster-logging-int.pem
    ```
 
-   Note: in development, if you don't have valid certs for these, you can just
+   > __NOTE:__: in development, if you don't have valid certs for these, you can just
    upload `localhost.pem` as a placeholder for each of these. This will avoid an
    error stemming from them not existing, but it will result in logging pods
-   crash looping in any clusters you make.
+   crash looping in any clusters you make. Additionally, no gateway resources are 
+   created in development so you should not need to execute the cert import statement
+   for the "-gwy" keyvault.
 
 1. In pre-production (int, e2e) certain certificates are provisioned via keyvault
 integration. These should be rotated and generated in the keyvault itself:
@@ -546,4 +544,4 @@ Development value: secrets/cluster-logging-int.pem
 ## Append Resource Group to Subscription Cleaner DenyList
 
 * We have subscription pruning that takes place routinely and need to add our resource group for the shared rp environment to the `denylist` of the cleaner:
-	* [https://github.com/Azure/ARO-RP/blob/e918d1b87be53a3b3cdf18b674768a6480fb56b8/hack/clean/clean.go#L29](https://github.com/Azure/ARO-RP/blob/e918d1b87be53a3b3cdf18b674768a6480fb56b8/hack/clean/clean.go#L29) 
+   * [https://github.com/Azure/ARO-RP/blob/e918d1b87be53a3b3cdf18b674768a6480fb56b8/hack/clean/clean.go#L29](https://github.com/Azure/ARO-RP/blob/e918d1b87be53a3b3cdf18b674768a6480fb56b8/hack/clean/clean.go#L29)