Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor Vault tests to allow more flexibility in mounting PKI and also be more representative of the integration experience of the user #1221

Merged
merged 11 commits into from
May 21, 2022
Merged

Refactor Vault tests to allow more flexibility in mounting PKI and also be more representative of the integration experience of the user #1221

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
604b50a
refactored TestVault
jmurret May 12, 2022
cdea857
Fixing name of CreateConnectCARootAndIntermediatePIKPolicy to CreateC…
jmurret May 12, 2022
89025ba
Modifying TestVault_TlsAutoReload
jmurret May 12, 2022
5a20b75
refactored all except WAN Fed has error
jmurret May 12, 2022
8af142c
fixing vault wan fed test
jmurret May 12, 2022
190732e
fixing snapshot agent vault test
jmurret May 12, 2022
f84ccf9
comment formatting
jmurret May 12, 2022
32f54f5
PR Feedback. Moving vault helper functions to be on the structs they…
jmurret May 18, 2022
c3eb0ca
update SaveVaultSecretConfiguration to be called KV2Secret
jmurret May 21, 2022
71ee4b1
PR Feedback. changing name of Save() onkv2secret to something morede…
jmurret May 21, 2022
e827ca7
PR feedback- replacing common roles and authmethod path with the use …
jmurret May 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
312 changes: 133 additions & 179 deletions acceptance/framework/vault/helpers.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,63 +6,11 @@ import (
"fmt"
"testing"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This diff is not the easiest to follow unfortunately. The best way to review this is to first gain an understanding of the vault helper functions are used in the tests. So look to the test files first and look for vault.* function calls.

"github.com/hashicorp/consul-k8s/acceptance/framework/config"
"github.com/hashicorp/consul-k8s/acceptance/framework/logger"
"github.com/hashicorp/go-uuid"
vapi "github.com/hashicorp/vault/api"
"github.com/stretchr/testify/require"
)

const (
gossipPolicy = `
path "consul/data/secret/gossip" {
capabilities = ["read"]
}`

tokenPolicyTemplate = `
path "consul/data/secret/%s" {
capabilities = ["read"]
}`

enterpriseLicensePolicy = `
path "consul/data/secret/license" {
capabilities = ["read"]
}`

// connectCAPolicy allows Consul to bootstrap all certificates for the service mesh in Vault.
// Adapted from https://www.consul.io/docs/connect/ca/vault#consul-managed-pki-paths.
connectCAPolicyTemplate = `
path "/sys/mounts" {
capabilities = [ "read" ]
}

path "/sys/mounts/connect_root" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}

path "/sys/mounts/%s/connect_inter" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}

path "/connect_root/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}

path "/%s/connect_inter/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
`
caPolicy = `
path "pki/cert/ca" {
capabilities = ["read"]
}`

snapshotAgentPolicy = `
path "consul/data/secret/snapshot-agent-config" {
capabilities = ["read"]
}`
)

// GenerateGossipSecret generates a random 32 byte secret returned as a base64 encoded string.
func GenerateGossipSecret() (string, error) {
// This code was copied from Consul's Keygen command:
Expand All @@ -80,177 +28,183 @@ func GenerateGossipSecret() (string, error) {
return base64.StdEncoding.EncodeToString(key), nil
}

// ConfigureGossipVaultSecret generates a gossip encryption key,
// stores it in Vault as a secret and configures a policy to access it.
func ConfigureGossipVaultSecret(t *testing.T, vaultClient *vapi.Client) string {
// Create the Vault Policy for the gossip key.
logger.Log(t, "Creating gossip policy")
err := vaultClient.Sys().PutPolicy("gossip", gossipPolicy)
require.NoError(t, err)

// Generate the gossip secret.
gossipKey, err := GenerateGossipSecret()
require.NoError(t, err)

// Create the gossip secret.
logger.Log(t, "Creating the gossip secret")
// ConfigurePKICerts configures roles in Vault so
// that controller webhook TLS certificates can be issued by Vault.
func ConfigurePKICerts(t *testing.T,
vaultClient *vapi.Client, baseUrl, allowedSubdomain, roleName, ns, datacenter,
maxTTL string) string {
allowedDomains := fmt.Sprintf("%s.consul,%s,%s.%s,%s.%s.svc", datacenter,
allowedSubdomain, allowedSubdomain, ns, allowedSubdomain, ns)
params := map[string]interface{}{
"data": map[string]interface{}{
"gossip": gossipKey,
},
"allowed_domains": allowedDomains,
"allow_bare_domains": "true",
"allow_localhost": "true",
"allow_subdomains": "true",
"generate_lease": "true",
"max_ttl": maxTTL,
}
_, err = vaultClient.Logical().Write("consul/data/secret/gossip", params)

_, err := vaultClient.Logical().Write(
fmt.Sprintf("%s/roles/%s", baseUrl, roleName), params)
require.NoError(t, err)

return gossipKey
}
certificateIssuePath := fmt.Sprintf("%s/issue/%s", baseUrl, roleName)
policy := fmt.Sprintf(`
path %q {
capabilities = ["create", "update"]
}`, certificateIssuePath)

// ConfigureEnterpriseLicenseVaultSecret stores it in Vault as a secret and configures a policy to access it.
func ConfigureEnterpriseLicenseVaultSecret(t *testing.T, vaultClient *vapi.Client, cfg *config.TestConfig) {
// Create the enterprise license secret.
logger.Log(t, "Creating the Enterprise License secret")
params := map[string]interface{}{
"data": map[string]interface{}{
"license": cfg.EnterpriseLicense,
},
}
_, err := vaultClient.Logical().Write("consul/data/secret/license", params)
// Create the server policy.
err = vaultClient.Sys().PutPolicy(roleName, policy)
require.NoError(t, err)

err = vaultClient.Sys().PutPolicy("license", enterpriseLicensePolicy)
require.NoError(t, err)
return certificateIssuePath
}

// ConfigureSnapshotAgentSecret stores it in Vault as a secret and configures a policy to access it.
func ConfigureSnapshotAgentSecret(t *testing.T, vaultClient *vapi.Client, cfg *config.TestConfig, config string) {
logger.Log(t, "Creating the Snapshot Agent Config secret in Vault")
params := map[string]interface{}{
"data": map[string]interface{}{
"config": config,
},
// ConfigurePKI generates a CA in Vault at a given path with a given policyName.
func ConfigurePKI(t *testing.T, vaultClient *vapi.Client, baseUrl, policyName, commonName string, skipPKIMount bool) {
if !skipPKIMount {
// Mount the PKI Secrets engine at the baseUrl.
mountErr := vaultClient.Sys().Mount(baseUrl, &vapi.MountInput{
Type: "pki",
Config: vapi.MountConfigInput{},
})
require.NoError(t, mountErr)
// Create root CA to issue Consul server certificates and the `consul-server` PKI role.
// See https://learn.hashicorp.com/tutorials/consul/vault-pki-consul-secure-tls.
// Generate the root CA.
params := map[string]interface{}{
"common_name": commonName,
"ttl": "24h",
}
_, mountErr = vaultClient.Logical().Write(fmt.Sprintf("%s/root/generate/internal", baseUrl), params)
require.NoError(t, mountErr)
}
_, err := vaultClient.Logical().Write("consul/data/secret/snapshot-agent-config", params)
require.NoError(t, err)

err = vaultClient.Sys().PutPolicy("snapshot-agent-config", snapshotAgentPolicy)
policy := fmt.Sprintf(`path "%s/cert/ca" {
capabilities = ["read"]
}`, baseUrl)
err := vaultClient.Sys().PutPolicy(policyName, policy)
require.NoError(t, err)
}

type KubernetesAuthRoleConfiguration struct {
ServiceAccountName string
KubernetesNamespace string
PolicyNames string
AuthMethodPath string
RoleName string
}

// ConfigureKubernetesAuthRole configures a role in Vault for the component for the Kubernetes auth method
// that will be used by the test Helm chart installation.
func ConfigureKubernetesAuthRole(t *testing.T, vaultClient *vapi.Client, consulReleaseName, ns, authPath, component, policies string) {
componentServiceAccountName := fmt.Sprintf("%s-consul-%s", consulReleaseName, component)

func (config *KubernetesAuthRoleConfiguration) ConfigureK8SAuthRole(t *testing.T, vaultClient *vapi.Client) {
// Create the Auth Roles for the component.
// Auth roles bind policies to Kubernetes service accounts, which
// then enables the Vault agent init container to call 'vault login'
// with the Kubernetes auth method to obtain a Vault token.
// Please see https://www.vaultproject.io/docs/auth/kubernetes#configuration
// for more details.
logger.Logf(t, "Creating the %q", componentServiceAccountName)
logger.Logf(t, "Creating the %q", config.ServiceAccountName)
jmurret marked this conversation as resolved.
Show resolved Hide resolved
params := map[string]interface{}{
"bound_service_account_names": componentServiceAccountName,
"bound_service_account_namespaces": ns,
"policies": policies,
"bound_service_account_names": config.ServiceAccountName,
"bound_service_account_namespaces": config.KubernetesNamespace,
"policies": config.PolicyNames,
"ttl": "24h",
}
_, err := vaultClient.Logical().Write(fmt.Sprintf("auth/%s/role/%s", authPath, component), params)
_, err := vaultClient.Logical().Write(fmt.Sprintf("auth/%s/role/%s", config.AuthMethodPath, config.RoleName), params)
require.NoError(t, err)
}

// ConfigureConsulCAKubernetesAuthRole configures a role in Vault that allows all service accounts
// within the installation namespace access to the Consul server CA.
func ConfigureConsulCAKubernetesAuthRole(t *testing.T, vaultClient *vapi.Client, ns, authPath string) {
// Create the CA role that all components will use to fetch the Server CA certs.
params := map[string]interface{}{
"bound_service_account_names": "*",
"bound_service_account_namespaces": ns,
"policies": "consul-ca",
"ttl": "24h",
}
_, err := vaultClient.Logical().Write(fmt.Sprintf("auth/%s/role/consul-ca", authPath), params)
require.NoError(t, err)
type PKIAndAuthRoleConfiguration struct {
ServiceAccountName string
BaseURL string
PolicyName string
RoleName string
CommonName string
CAPath string
CertPath string
KubernetesNamespace string
DataCenter string
MaxTTL string
AuthMethodPath string
AllowedSubdomain string
SkipPKIMount bool
}

// ConfigurePKICA generates a CA in Vault.
func ConfigurePKICA(t *testing.T, vaultClient *vapi.Client) {
// Create root CA to issue Consul server certificates and the `consul-server` PKI role.
// See https://learn.hashicorp.com/tutorials/consul/vault-pki-consul-secure-tls.
// Generate the root CA.
params := map[string]interface{}{
"common_name": "Consul CA",
"ttl": "24h",
func (config *PKIAndAuthRoleConfiguration) ConfigurePKIAndAuthRole(t *testing.T, vaultClient *vapi.Client) {
config.CAPath = fmt.Sprintf("%s/cert/ca", config.BaseURL)
// Configure role with read access to <baseURL>/cert/ca
ConfigurePKI(t, vaultClient, config.BaseURL, config.PolicyName,
config.CommonName, config.SkipPKIMount)
// Configure role with create and update access to issue certs at
// <baseURL>/issue/<roleName>
config.CertPath = ConfigurePKICerts(t, vaultClient, config.BaseURL,
config.AllowedSubdomain, config.PolicyName, config.KubernetesNamespace,
config.DataCenter, config.MaxTTL)
// Configure AuthMethodRole that will map the service account name
// to the Vault role
authMethodRoleConfig := &KubernetesAuthRoleConfiguration{
ServiceAccountName: config.ServiceAccountName,
KubernetesNamespace: config.KubernetesNamespace,
AuthMethodPath: config.AuthMethodPath,
RoleName: config.RoleName,
PolicyNames: config.PolicyName,
}
_, err := vaultClient.Logical().Write("pki/root/generate/internal", params)
require.NoError(t, err)

err = vaultClient.Sys().PutPolicy("consul-ca", caPolicy)
require.NoError(t, err)
authMethodRoleConfig.ConfigureK8SAuthRole(t, vaultClient)
}

// ConfigurePKICertificates configures roles in Vault so that Consul server TLS certificates
// can be issued by Vault.
func ConfigurePKICertificates(t *testing.T, vaultClient *vapi.Client, consulReleaseName, ns, datacenter string, maxTTL string) string {
consulServerDNSName := consulReleaseName + "-consul-server"
allowedDomains := fmt.Sprintf("%s.consul,%s,%s.%s,%s.%s.svc", datacenter, consulServerDNSName, consulServerDNSName, ns, consulServerDNSName, ns)
params := map[string]interface{}{
"allowed_domains": allowedDomains,
"allow_bare_domains": "true",
"allow_localhost": "true",
"allow_subdomains": "true",
"generate_lease": "true",
"max_ttl": maxTTL,
}

pkiRoleName := fmt.Sprintf("server-cert-%s", datacenter)

_, err := vaultClient.Logical().Write(fmt.Sprintf("pki/roles/%s", pkiRoleName), params)
require.NoError(t, err)

certificateIssuePath := fmt.Sprintf("pki/issue/%s", pkiRoleName)
serverTLSPolicy := fmt.Sprintf(`
path %q {
capabilities = ["create", "update"]
}`, certificateIssuePath)

// Create the server policy.
err = vaultClient.Sys().PutPolicy(pkiRoleName, serverTLSPolicy)
require.NoError(t, err)

return certificateIssuePath
type SaveVaultSecretConfiguration struct {
Path string
Key string
PolicyName string
Value string
}

// ConfigureACLTokenVaultSecret generates a token secret ID for a given name,
// stores it in Vault as a secret and configures a policy to access it.
func ConfigureACLTokenVaultSecret(t *testing.T, vaultClient *vapi.Client, tokenName string) string {
// Create the Vault Policy for the token.
logger.Logf(t, "Creating %s token policy", tokenName)
policyName := fmt.Sprintf("%s-token", tokenName)
tokenPolicy := fmt.Sprintf(tokenPolicyTemplate, tokenName)
err := vaultClient.Sys().PutPolicy(policyName, tokenPolicy)
require.NoError(t, err)

// Generate the token secret.
token, err := uuid.GenerateUUID()
func (config *SaveVaultSecretConfiguration) Save(t *testing.T, vaultClient *vapi.Client) {
jmurret marked this conversation as resolved.
Show resolved Hide resolved
policy := fmt.Sprintf(`
path "%s" {
capabilities = ["read"]
}`, config.Path)
// Create the Vault Policy for the gossip key.
jmurret marked this conversation as resolved.
Show resolved Hide resolved
logger.Log(t, "Creating policy")
err := vaultClient.Sys().PutPolicy(config.PolicyName, policy)
require.NoError(t, err)

// Create the replication token secret.
logger.Logf(t, "Creating the %s token secret", tokenName)
// Create the gossip secret.
jmurret marked this conversation as resolved.
Show resolved Hide resolved
logger.Logf(t, "Creating the %s secret", config.Path)
params := map[string]interface{}{
"data": map[string]interface{}{
"token": token,
config.Key: config.Value,
},
}
_, err = vaultClient.Logical().Write(fmt.Sprintf("consul/data/secret/%s", tokenName), params)
_, err = vaultClient.Logical().Write(config.Path, params)
require.NoError(t, err)

return token
}

// CreateConnectCAPolicy creates the Vault Policy for the connect-ca in a given datacenter.
func CreateConnectCAPolicy(t *testing.T, vaultClient *vapi.Client, datacenter string) {
// CreateConnectCAPolicyForDatacenter creates the Vault Policy for the connect-ca in a given datacenter.
jmurret marked this conversation as resolved.
Show resolved Hide resolved
func CreateConnectCARootAndIntermediatePKIPolicy(t *testing.T, vaultClient *vapi.Client, policyName, rootPath, intermediatePath string) {
// connectCAPolicy allows Consul to bootstrap all certificates for the service mesh in Vault.
// Adapted from https://www.consul.io/docs/connect/ca/vault#consul-managed-pki-paths.
connectCAPolicyTemplate := `
path "/sys/mounts" {
capabilities = [ "read" ]
}
path "/sys/mounts/%s" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
path "/sys/mounts/%s" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
path "/%s/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
path "/%s/*" {
capabilities = [ "create", "read", "update", "delete", "list" ]
}
`
err := vaultClient.Sys().PutPolicy(
fmt.Sprintf("connect-ca-%s", datacenter),
fmt.Sprintf(connectCAPolicyTemplate, datacenter, datacenter))
policyName,
fmt.Sprintf(connectCAPolicyTemplate, rootPath, intermediatePath, rootPath, intermediatePath))
jmurret marked this conversation as resolved.
Show resolved Hide resolved
require.NoError(t, err)
}
7 changes: 0 additions & 7 deletions acceptance/framework/vault/vault_cluster.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,13 +147,6 @@ func (v *VaultCluster) bootstrap(t *testing.T, vaultNamespace string) {
})
require.NoError(t, err)

// Enable the PKI Secrets engine.
err = v.vaultClient.Sys().Mount("pki", &vapi.MountInput{
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has moved to ConfigurePKI() in the above helpers file because we will need to mount more than one PKI engine and it will not always be at pki base path.

Type: "pki",
Config: vapi.MountConfigInput{},
})
require.NoError(t, err)

namespace := v.helmOptions.KubectlOptions.Namespace
vaultServerServiceAccountName := fmt.Sprintf("%s-vault", v.releaseName)
v.ConfigureAuthMethod(t, v.vaultClient, "kubernetes", "https://kubernetes.default.svc", vaultServerServiceAccountName, namespace)
Expand Down
2 changes: 1 addition & 1 deletion acceptance/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ require (
github.com/hashicorp/consul-k8s/control-plane v0.0.0-20211207212234-aea9efea5638
github.com/hashicorp/consul/api v1.12.0
github.com/hashicorp/consul/sdk v0.9.0
github.com/hashicorp/go-uuid v1.0.2
github.com/hashicorp/go-uuid v1.0.3
github.com/hashicorp/vault/api v1.2.0
github.com/stretchr/testify v1.7.0
gopkg.in/yaml.v2 v2.4.0
Expand Down
Loading