Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Removes unused values from helm chart #29

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Removes unused values from helm chart #29

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
bec9b21
Sets default values for GitEndpoint RepoNeedsKey
ashtongraves Nov 1, 2023
33cf941
Removes dashboard
ashtongraves Nov 1, 2023
7b844c2
Removes HostKeySecret and HostCertSecret
ashtongraves Nov 1, 2023
305f7f3
Removes Slate value section and slate references
ashtongraves Nov 7, 2023
a725e33
Bumps chart version to 4.7.0
ashtongraves Nov 7, 2023
3bf3c9a
Adds ProxyJumpUser
ashtongraves May 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion supported/osg-htc/osg-hosted-ce/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ apiVersion: v1
appVersion: "V5-branch"
description: OSG Hosted Compute Entrypoint
name: osg-hosted-ce
version: 4.6.3
version: 4.7.0
15 changes: 1 addition & 14 deletions supported/osg-htc/osg-hosted-ce/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,12 +242,6 @@ To avoid these rate limits, it's possible to bootstrap the certificate request p

slate secret create <YOUR HOST KEY SECRET NAME> --cluster <YOUR CLUSTER> --group <YOUR GROUP> --from-file host.key=<PATH TO KEYFILE>

1. Update your values file to use the host key secret that you've created:

HostCredentials:
HostKeySecret: <YOUR HOST KEY SECRET NAME>
HostCertSecret: null

1. Upon successful startup of the Hosted CE app, the Let's Encrypt host certificate can be found in the instance logs:

slate instance logs <YOUR INSTANCE NAME> --container osg-hosted-ce --max-lines 0
Expand All @@ -258,12 +252,6 @@ To avoid these rate limits, it's possible to bootstrap the certificate request p

slate secret create <YOUR HOST CERT SECRET NAME> --cluster <YOUR CLUSTER> --group <YOUR GROUP> --from-file host.cert=<PATH TO CERT FILE>

1. Update your values file to use the host key secret that you've created:

HostCredentials:
HostKeySecret: <YOUR HOST KEY SECRET NAME>
HostCertSecret: <YOUR HOST CERT SECRET NAME>

### Developer
Simply disable this. It is in place for the purpose of OSG Internal Testbed hosts, and is not intended for use with production CEs.

Expand Down Expand Up @@ -329,8 +317,7 @@ BoscoOverrides:
GitKeySecret: null

HostCredentials:
HostKeySecret: null
HostCertSecret: null
HostCertKeySecret: null

Developer:
Enabled: false
Expand Down
4 changes: 0 additions & 4 deletions supported/osg-htc/osg-hosted-ce/templates/_helpers.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,6 @@ If release name contains chart name it will be used as a full name.
{{- end -}}
{{- end -}}

{{- define "namespace" -}}
{{- .Release.Namespace | trimPrefix "slate-vo-" | printf " %s" -}}
{{- end -}}

{{/*
Create chart name and version as used by the chart label.
*/}}
Expand Down
7 changes: 4 additions & 3 deletions supported/osg-htc/osg-hosted-ce/templates/configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,8 @@ data:
{{ end }}

SLATE_HOSTED_CE = True
SCHEDD_ATTRS = $(SCHEDD_ATTRS) SLATE_HOSTED_CE
OSG_HOSTED_CE = True
ashtongraves marked this conversation as resolved.
Show resolved Hide resolved
SCHEDD_ATTRS = $(SCHEDD_ATTRS) SLATE_HOSTED_CE OSG_HOSTED_CE

# Templatize IDTOKEN generation for glidein -> CE collector advertising (SOFTWARE-5556)
{{- range $index, $map := .Values.SciTokenRemoteUserMapping }}
Expand Down Expand Up @@ -136,15 +137,15 @@ data:
apiVersion: v1
kind: ConfigMap
metadata:
name: osg-hosted-ce-{{ .Values.Instance }}-slate-scitokens
name: osg-hosted-ce-{{ .Values.Instance }}-scitokens
labels:
app: osg-hosted-ce
instance: {{ .Values.Instance }}
release: {{ .Release.Name }}
app.kubernetes.io/part-of: {{ .Chart.Name }}
app.kubernetes.io/instance: {{ .Release.Name }}
data:
50-slate-scitokens.conf: |+
50-scitokens.conf: |+
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth looking in the container to make sure there isn't anything hardcoded referencing this file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I searched here: https://github.com/search?q=repo%3Ahtcondor%2Fhtcondor-ce%20scitokens.conf&type=code
And here: https://github.com/search?q=repo%3Aopensciencegrid%2Fdocker-compute-entrypoint%20scitokens.conf&type=code

And don't see any reference at all to this. Is there somewhere else I'm supposed to check? Also what's the difference between that conf and this one: https://github.com/htcondor/htcondor-ce/blob/056e76af28d258db82dc895c0b6a82c02e765fa6/config/mapfiles.d/10-scitokens.conf

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we're good as the CE container images look at the entire mapfiles.d dir for mappings to determine which users need to be provisioned: https://github.com/opensciencegrid/docker-compute-entrypoint/blob/master/base/etc/osg/image-config.d/ce-common-startup

The default conf that you link is mostly intended for admins setting up their own local CE and notice that there aren't any uncommented lines. The entire mapfiles.d dir is parsed in alphanumeric order and concatenated together to get credential -> user mappings (the file that determines what's included is /etc/condor-ce/mapfiles.d).

You'll see this pattern used a lot for configuration, e.g. conf.d dirs, across different Linux applications.

{{- range $index, $map := .Values.SciTokenRemoteUserMapping }}
{{- range $url, $user := $map }}
SCITOKENS /^{{ $url | replace "/" "\\/" | replace "." "\\." | replace "-" "\\-" }}{{ if not ( $url | contains "," ) }},{{ end }}/ {{ $user }}
Expand Down
52 changes: 5 additions & 47 deletions supported/osg-htc/osg-hosted-ce/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ metadata:
app: {{ template "osg-hosted-ce.name" . }}
release: {{ .Release.Name }}
instance: {{ .Values.Instance }}
instanceID: {{ .Values.SLATE.Instance.ID | quote }}
app.kubernetes.io/part-of: {{ .Chart.Name }}
app.kubernetes.io/instance: {{ .Release.Name }}
spec:
Expand Down Expand Up @@ -47,9 +46,9 @@ spec:
- name: osg-hosted-ce-{{ .Values.Instance }}-htcondor-ce-configuration
configMap:
name: osg-hosted-ce-{{ .Values.Instance }}-htcondor-ce-configuration
- name: osg-hosted-ce-{{ .Values.Instance }}-slate-scitokens
- name: osg-hosted-ce-{{ .Values.Instance }}-scitokens
configMap:
name: osg-hosted-ce-{{ .Values.Instance }}-slate-scitokens
ashtongraves marked this conversation as resolved.
Show resolved Hide resolved
name: osg-hosted-ce-{{ .Values.Instance }}-scitokens
- name: bosco-ssh-private-key-volume
secret:
secretName: {{ .Values.RemoteCluster.PrivateKeySecret }}
Expand Down Expand Up @@ -82,25 +81,6 @@ spec:
- key: tls.key
path: hostkey.pem
mode: 0o400
{{ else }}
{{ if and .Values.HostCredentials.HostCertSecret .Values.HostCredentials.HostKeySecret }}
- name: osg-hosted-ce-hostcert-volume
secret:
secretName: {{ .Values.HostCredentials.HostCertSecret }}
items:
- key: host.cert
path: hostcert.pem
mode: 256
{{ end }}
{{ if .Values.HostCredentials.HostKeySecret }}
- name: osg-hosted-ce-hostkey-volume
secret:
secretName: {{ .Values.HostCredentials.HostKeySecret }}
items:
- key: host.key
path: hostkey.pem
mode: 256
{{ end }}
{{ end }}
{{ if .Values.BoscoOverrides.Enabled }}
{{ if .Values.BoscoOverrides.GitKeySecret }}
Expand Down Expand Up @@ -152,17 +132,6 @@ spec:
{{ if .Values.HostCredentials.HostCertKeySecret }}
- name: osg-hosted-ce-hostcertkey-volume
mountPath: /etc/grid-security-orig.d
{{ else }}
{{ if and .Values.HostCredentials.HostCertSecret .Values.HostCredentials.HostKeySecret }}
- name: osg-hosted-ce-hostcert-volume
mountPath: /etc/grid-security/hostcert.pem
subPath: hostcert.pem
{{ end }}
{{ if .Values.HostCredentials.HostKeySecret }}
- name: osg-hosted-ce-hostkey-volume
mountPath: /etc/grid-security/hostkey.pem
subPath: hostkey.pem
{{ end }}
{{ end }}
{{ if .Values.Persistence.LogVolume}}
- name: log-volume
Expand All @@ -176,21 +145,14 @@ spec:
{{ end }}
{{ end }}
{{ if .Values.SciTokenRemoteUserMapping }}
- name: osg-hosted-ce-{{ .Values.Instance }}-slate-scitokens
mountPath: /etc/condor-ce/mapfiles.d/50-slate-scitokens.conf
subPath: 50-slate-scitokens.conf
- name: osg-hosted-ce-{{ .Values.Instance }}-scitokens
mountPath: /etc/condor-ce/mapfiles.d/50-scitokens.conf
subPath: 50-scitokens.conf
{{ end }}
ashtongraves marked this conversation as resolved.
Show resolved Hide resolved
ports:
- name: htcondor-ce
containerPort: 9619
protocol: TCP
{{ if or .Values.Dashboard.Enabled
( and (not .Values.HostCredentials.HostCertKeySecret)
(not .Values.HostCredentials.HostCertSecret) ) }}
- name: http # needed to get a cert issued from Let's Encrypt
containerPort: 80
protocol: TCP
{{ end }}
env:
{{ if eq .Values.Networking.ServiceType "HostNetwork" }}
- name: _CONDOR_NETWORK_HOSTNAME
Expand All @@ -215,10 +177,6 @@ spec:
{{ else }}
value: "false"
{{ end }}
{{ if .Values.Dashboard.Enabled }}
- name: ENABLE_CE_VIEW
value: "true"
{{ end }}
{{ if .Values.Debug.ContinueOnError }}
- name: HOSTED_CE_CONTINUE_ON_ERROR
value: "true"
Expand Down
6 changes: 0 additions & 6 deletions supported/osg-htc/osg-hosted-ce/templates/networkpolicy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,5 @@ spec:
- ports:
- protocol: TCP
port: 9619
{{ if or .Values.Dashboard.Enabled
( and (not .Values.HostCredentials.HostCertKeySecret)
(not .Values.HostCredentials.HostCertSecret) ) }}
- protocol: TCP
port: 80
{{ end }}
egress:
- {}
8 changes: 0 additions & 8 deletions supported/osg-htc/osg-hosted-ce/templates/service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,6 @@ spec:
port: 9619
targetPort: htcondor-ce
protocol: TCP
{{ if or .Values.Dashboard.Enabled
( and (not .Values.HostCredentials.HostCertKeySecret)
(not .Values.HostCredentials.HostCertSecret) ) }}
- name: http # needed to get a cert issued from Let's Encrypt (since the admin's not supplying a cert/key as a secret)
port: 80
targetPort: http
protocol: TCP
{{ end }}
{{ if .Values.Networking.RequestIP }}
loadBalancerIP: {{ .Values.Networking.RequestIP }}
{{ end }}
Expand Down
34 changes: 6 additions & 28 deletions supported/osg-htc/osg-hosted-ce/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ RemoteCluster:
# login04.osgconnect.net ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCqsciY/FmcH6yedW7DxbTwLjkwlgVcbcG43QwdaSvMuN5EHAeLRH2P6ig3H6s/4hZqJn7AU2oNsoIqjZZ8iHqGyiTB9r70Aovt9PUuQXw5qHvYiIoyw49/waJ+yzu4+UD3qfnxeHuRw5EOaXia72MNJJ4lBMoS6iw1JveFJY4rBgKZ7nta1eAGGfHADsf5lfhlIXnJPN050X6zcUjqEEKXjdUkFa3wDiVbFEIESCMwi1b6Q1OXGGFwuSNse1X38CREBzQ+NjOB97o70chZoh2jZ2O3iZZZbujYcDzpxF3RxmewFC3pDmkoOpJrfV2W58n1KdJJnyvjlqEZGXCXcAlB
# login04.osgconnect.net ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCnySArMSVd0O3UniYN4vX3a/d9d9mEMnrf2z1S900t0GrxVazVGu3ObeLfWwYRc3qFXNRPGh3mKcz5QVG6gcm4=
# login04.osgconnect.net ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJBW6weAaFSdLYksn4vIvqt4+geNVQ/bUvp71kTeQsKV
# SLATE secret with 'bosco.key' containing the SSH key to access LoginHost:
# Secret with 'bosco.key' containing the SSH key to access LoginHost:
PrivateKeySecret: lincolnb-bosco
# Secret containing a signed certificate (required by some sites)
# See CertificateFile in `man ssh_config`
Expand Down Expand Up @@ -144,9 +144,7 @@ SciTokenRemoteUserMapping:
# max_wall_time = 34560
# queue = opportunistic

# Enable persistence to HostedCE files by creating corresponding SLATE Volumes
# SLATE Creates a PVC in Kubernetes which can be reclaimed on restart
# The value for each volume should correspond to the volume name within SLATE
# Enable persistence to HostedCE files by creating corresponding Volumes
Persistence:
# /var/log/condor-ce
LogVolume: null
Expand All @@ -160,8 +158,9 @@ Persistence:
BoscoOverrides:
Enabled: false
BoscoOverrideDir: null
GitEndpoint: https://github.com/slateci/bosco-override-template
# If GitEndpoint requires authentication, create a SLATE secret with
GitEndpoint: [email protected]:opensciencegrid/hosted-ce-config.git
RepoNeedsPrivKey: true
# If GitEndpoint requires authentication, create a secret with
# 'git.key' containing the private SSH key that can access
# it. Specify the name of the secret in GitKeySecret:
GitKeySecret: null
Expand All @@ -188,28 +187,13 @@ ServiceAnnotations: {}

HostCredentials:
# Name of the secret containing a host key and certificate in
# "tls.key" and "tls.crt", respectively. If defined, values of
# HostCertSecret and HostKeySecret are ignored.
# "tls.key" and "tls.crt", respectively.
HostCertKeySecret: null
# Use a pre-existing host key to request a new Let's Encrypt
# certificate If HostCertSecret is also specified, the Let's Encrypt
# request is skipped. Secret must contain a "host.key" key
# containing the encoded host key.
HostKeySecret: null
# Use a pre-existing host certificate instead of requesting a new
# Let'S Encrypt certificate. If HostKeySecret is not specified, a
# new Let's Encrypt certificate and key are requested anyway.
# Secret must contain a "host.cert" containing the encoded host
# certificate.
HostCertSecret: null
# If set to 'true', use the Let's Encrypt staging server. This is
# useful for avoiding Let's Encrypt rate limits when first setting
# up a CE. NOT SUITABLE FOR PRODUCTION USE.
LetsEncryptStaging: false

Dashboard:
Enabled: false

# Choose which tag to use for the specified containers
ContainerTags:
HostedCE: release
Expand All @@ -225,9 +209,3 @@ Debug:
# - Generates a test CA and self-signed host cert/key pair
Developer:
Enabled: false

### SLATE-START ###
SLATE:
Instance:
ID: "untagged"
### SLATE-END ###