-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix Solr Auth Binding #29
Conversation
generate-env.sh now works whether you're using a KinD cluster or a real one in your kubeconfig
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm still unclear on what the fix actually was... Curling from inside the Solr container? Changing the default Solr image in use? Please explain!
…s to only be in 8.11?
e1c9ca5
to
736fe2d
Compare
I reverted the variable in question.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mogul I don't know what caused the 'binding' process to fail prior to this PR. There were a lot of changes to this repo in this PR that may have effected it. Kubernetes token issues. Curl unable to reach API. EKS vs. KiND peculiarities. Either way, the repo has now been tested with KiND and with a real EKS cluster, so I have confidence that it will work this time.
TODO: Document how to manually test repo with a real EKS cluster. The steps are scattered throughout this repo and GSA/data.gov#3523. It just needs to be consolidated.
Related Repo Update: curl
was installed in our base solr image https://github.com/GSA/catalog.data.gov/blob/feature/solr8/solr/Dockerfile#L11-L12
@@ -5,7 +5,8 @@ | |||
"service_name": "solr-cloud", | |||
"service_id": "b9013a91-9ce8-4c18-8035-a135a8cd6ff9", | |||
"plan_id": "e35e9675-413f-4f42-83de-ad5003357e77", | |||
"provision_params": {"solrJavaMem":"-Xms300m -Xmx300m", "solrMem":"1G", "solrCpu":"1000m", "cloud_name":"demo"}, | |||
"provision_params": {"solrJavaMem":"-Xms300m -Xmx300m", "solrMem":"1G", "solrCpu":"1000m", "cloud_name":"demo", "solrImageRepo":"ghcr.io/gsa/catalog.data.gov.solr", "solrImageTag":"8-curl"}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use the new image with curl for tests.
CURRENT_CONTEXT=$(kubectl config current-context) | ||
SOLR_CLUSTER_CA_CERTIFICATE=$(kubectl config view --raw -o json | jq -r '.clusters[]| select(.name | contains("'${CURRENT_CONTEXT}'")) .cluster["certificate-authority-data"]') | ||
SOLR_TOKEN=$(kubectl get secret $( kubectl get serviceaccount default -n default -o json | jq -r '.secrets[0].name' ) -n default -o json | jq -r .data.token) | ||
CURRENT_CLUSTER=$(kubectl config view --raw -o json | jq -r '.contexts[]| select(.name | contains("'"${CURRENT_CONTEXT}"'")) .context.cluster') | ||
CURRENT_USER=$(kubectl config view --raw -o json | jq -r '.contexts[]| select(.name | contains("'"${CURRENT_CONTEXT}"'")) .context.user') | ||
SOLR_CLUSTER_CA_CERTIFICATE=$(kubectl config view --raw -o json | jq -r '.clusters[]| select(.name | contains("'"${CURRENT_CLUSTER}"'")) .cluster["certificate-authority-data"]') | ||
SOLR_TOKEN=$(echo -n `kubectl config view --raw -o json | jq -r '.users[]| select(.name | contains("'"${CURRENT_USER}"'")) .user["token"]'` | base64 -w 0) | ||
SOLR_SERVER=$(kubectl config view --raw -o json | jq -r '.clusters[]| select(.name | contains("'"${CURRENT_CLUSTER}"'")) .cluster["server"]') | ||
|
||
# We need the Docker-internal control plane URL to be resolved for the CSB | ||
# when running in a container | ||
SOLR_DOCKER_SERVER=$(kind get kubeconfig --internal --name=$(kind get clusters | grep datagov-broker-test) | grep server | cut -d ' ' -f 6-) | ||
SOLR_DOMAIN_NAME=${SOLR_DOMAIN_NAME:-ing.local.domain} | ||
|
||
# We need the localhost control plan URL to be used for direct access when we | ||
# work outside the CSB | ||
SOLR_LOCALHOST_SERVER=$(kind get kubeconfig --name=$(kind get clusters | grep datagov-broker-test) | grep server | cut -d ' ' -f 6-) | ||
if [[ "${CURRENT_CLUSTER}" == "kind-datagov-broker-test" ]]; then | ||
# If the test cluster is in KinD we need the CSB to use | ||
# a control plane URL resolvable from inside the CSB Docker container | ||
CURRENT_USER=kind-datagov-broker-test | ||
SOLR_CP_SERVER=$(kind get kubeconfig --internal --name="$(kind get clusters | grep datagov-broker-test)" | grep server | cut -d ' ' -f 6-) | ||
SOLR_TOKEN=$(kubectl get secret $(kubectl get secrets | grep -oh "default-token-[a-z]*\s") -o json | jq .data.token | tr -d '"') | ||
if [[ "$SOLR_TOKEN" == "null" ]]; then | ||
# The format of the secret is different if there are more than one token associated with a secret. | ||
# The first token works reliably | ||
SOLR_TOKEN=$(kubectl get secret $(kubectl get secrets | grep -oh "default-token-[a-z]*\s") -o json | jq .items[0].data.token | tr -d '"') | ||
fi | ||
else | ||
# Otherwise it's the same as the normal server control plane URL | ||
SOLR_CP_SERVER=${SOLR_SERVER} | ||
fi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Set up repo to cooperate with either a KiND cluster or real EKS cluster.
@@ -70,23 +70,20 @@ resource "null_resource" "manage_solr_user" { | |||
# Can't reuse containers because they are left in an unpredictable state after a single run | |||
# Wait for the command to run before deleting the container | |||
command = <<-EOF | |||
kubectl --kubeconfig <(echo $KUBECONFIG | base64 -d) run temp1 --image=curlimages/curl -- \ | |||
kubectl --kubeconfig <(echo $KUBECONFIG | base64 -d) exec ${self.triggers.cloud_name}-solrcloud-0 -- curl \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
curl
to Solr API is now performed from within the solrcloud pod itself
@@ -45,6 +45,7 @@ resource "helm_release" "solrcloud" { | |||
"replicas" = var.replicas # How many replicas you want | |||
"solrOptions.javaMemory" = var.solrJavaMem # How much memory to give each replica | |||
"solrOptions.security.authenticationType" = "Basic" | |||
"ingressOptions.annotations.nginx\\.ingress\\.kubernetes\\.io/proxy-body-size" = "999m" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Port update from earlier release, 574336a#diff-ba41f8c4953fba843cf840ae370b5681502661992bf51cede0ea49d92187a6aeR45
Related to GSA/data.gov#3523