contrib: Dump keys/certs from acme.json to files #1484

brianredbeard · 2017-04-23T19:06:55Z

In the event that a user needs to explode their acme.json file into
a set of directories and relevant files for troubleshooting or use
with other programs this script will parse them into the components
in the following path structure:

certdir
├── certs
│   ├── domain-1.example.com
│   ├── domain-2.example.com
│   └── domain-n.example.com
└── private
    └── letsencrypt.key

timoreimann

Thanks for the PR! I left a few comments.

timoreimann · 2017-04-23T19:24:11Z

contrib/scripts/dumpcerts.sh

@@ -0,0 +1,141 @@
+#!/bin/bash


/usr/bin/env bash is more portable and thus preferred.

timoreimann · 2017-04-23T19:26:36Z

contrib/scripts/dumpcerts.sh

+# 4 - The destination certificate directory does not exist
+# 8 - Missing private key
+
+USAGE="dumpcerts.sh <path to acme> <destination cert directory>"


Can we use $(basename "$0") to display the script name in order to be portable should we ever rename the file?

timoreimann · 2017-04-23T19:28:27Z

contrib/scripts/dumpcerts.sh

+# 1 - A component is missing or could not be read
+# 2 - There was a problem reading acme.json
+# 4 - The destination certificate directory does not exist
+# 8 - Missing private key


Which exit codes one is allowed to use is a bit disputed; personally, I'm okay with using the "reserved" ones in the low-digit range.

Why powers of two though instead of just 1, 2, 3, and 4? We don't really need to shift numbers around, do we?

This would just be so that in the long run you could continue through a number of failure situations and then give the user a composite at the end which could be used to identify all of the things that have gone wrong before doing anything (potentially) destructive.

timoreimann · 2017-04-23T19:31:10Z

contrib/scripts/dumpcerts.sh

+	echo ""
+	echo "You must have the binary 'jq' to use this."
+	echo "jq is available at: https://stedolan.github.io/jq/download/"
+	echo ""


Please output the message to stderr (>&2) using either a here document or a string stretching over multiple lines (SO answer).

Same for other error outputs below.

timoreimann · 2017-04-23T19:33:58Z

contrib/scripts/dumpcerts.sh

+USAGE="dumpcerts.sh <path to acme> <destination cert directory>"
+
+# Allow us to exit on a missing jq binary
+exitJQ() {


I'd prefer a naming convention using only lower-case and underscores (exit_jq), like the Google style guide recommends as well.

timoreimann · 2017-04-23T19:50:58Z

contrib/scripts/dumpcerts.sh

+# after writing the private key switch it back to the default
+oldumask=$(umask)
+umask 177
+# For some reason traefik stores the private key in stripped base64 format, but


It'd be nice to know the reason and phrase it here.

Maybe @emilevauge or @dtomcej know?

@timoreimann what is your point exactly ? base64 ?

@emilevauge Yes. I think it would be nice to rephrase

For some reason traefik stores the private key in stripped base64 format

into something like

traefik stores the private key in stripped base64 format because [reason]

or just

traefik stores the private key in stripped base64 format (without explanation)

I don't think we should leave comments in scripts that express uncertainty about the reasoning of our own code if we can avoid it.

timoreimann · 2017-04-23T19:52:53Z

contrib/scripts/dumpcerts.sh

+}
+
+
+acme="${1}"


I'd readonly the variable.

timoreimann · 2017-04-23T19:52:56Z

contrib/scripts/dumpcerts.sh

+
+
+acme="${1}"
+certdir="${2}"


I'd readonly the variable.

timoreimann · 2017-04-23T19:53:45Z

contrib/scripts/dumpcerts.sh

+# and place each in a variable for later use, normalizing the path
+mkdir -p "${certdir}"/{certs,private}
+
+pdir=$(realpath "${certdir}/private/")


Any particular reason we need to use realpath? AFAIK, it's not available on all systems.

Was only really using this to normalize paths in the event of ${certdir} being provided with a trailing slash. I was of two minds about this. realpath is a part of coreutils and thus should be on the lion's share of hosts (at least where traefik would be able to run). On the other hand it is a better answer to just guarantee this at the time of assignment via ${certdir%/}... Admittedly this was me making this mostly for myself and thinking of it as "contrib" status work (YMMV). Regardless, will fix with the assignment.

timoreimann · 2017-04-23T19:59:06Z

contrib/scripts/dumpcerts.sh

+# Save the existing umask, change the default mode to 600, then
+# after writing the private key switch it back to the default
+oldumask=$(umask)
+umask 177


Shouldn't we use a TRAP to revert the umask in order to cover for the case where the script breaks anywhere between this line and the point where we return to the old umask?

emilevauge · 2017-04-24T15:44:53Z

@brianredbeard Wow great 👍
I think it would be even better if we could add a new command to Traefik (like traefik exportACME /etc/ssl). It would allow to export certs even in cluster mode (certs from the KV store). WDYT ?

brianredbeard · 2017-04-25T06:31:40Z

@emilevauge More than likely this would be a good idea, that being said this was a case of something that I was solving for myself anyways and figured could be useful for others. I'd say making something like this available until a better answer is available (and then burning this script in a fire) is more than acceptable.

ldez · 2017-04-29T20:36:31Z

@brianredbeard It's clearly acceptable, can you fix the comments made by @timoreimann ? Thanks.

@emilevauge I've opened the issue #1512 for make the command in Traefik.

emilevauge · 2017-05-19T14:07:11Z

Ping @brianredbeard

In the event that a user needs to explode their acme.json file into a set of directories and relevant files for troubleshooting or use with other programs this script will parse them into the components in the following path structure: ``` certdir ├── certs │ ├── domain-1.example.com │ ├── domain-2.example.com │ └── domain-n.example.com └── private └── letsencrypt.key ```

timoreimann · 2017-06-22T22:54:40Z

@brianredbeard I carried your PR "in-place" and addressed all of my own comments that seemed undisputed. Hope that's okay with you. Feel free to comment back on me, of course.

@emilevauge PTAL.

brianredbeard · 2017-06-23T06:29:05Z

That's absolutely fine by me! Sorry about being a drag ass but you have my full support.

ldez · 2017-06-23T23:09:46Z

Due to SemaphoreCI, I close and reopen the PR.

emilevauge

Thanks @brianredbeard and @timoreimann ❤️
LGTM

kachkaev · 2017-07-01T09:08:01Z

Will this dump the certs automatically in real time? I'd like to mount a volume with my certs into gitlab and a mailserver, so am looking to find a folder with pem somewhere.

frederik · 2017-09-02T11:04:54Z

hi @brianredbeard. First of all, thanks for the contribution.

I have another (quick) follow up question. From how I understand let's encrypt it has two certificates. This script is extracting the domain handling key into a .key file and the certificate into the .pem file. But it doesn't extract the private key for the individual certificate (i.e. DomainsCertificate.Certs[n].Certificate.PrivateKey).

Gitlab Container Registry is asking me to provide both, so I did the base64 extraction manually to confirm that this was actually required.

I suppose extracting the PrivateKey would only be a minor duplication in the script.

tl;dr

Is there a reason you chose not to extract the PrivateKey? (or just didn't require it?)

jujes · 2017-09-22T12:16:59Z

someone know about it:

dumpcerts.sh: 40: set: Illegal option -o pipefail

kachkaev · 2017-09-22T13:10:08Z

@frederik could you please share your current workaround? I might need to pass a private key to gitlab or another upstream service in near future :–)

frederik · 2017-09-22T16:09:09Z

I took the contents from DomainsCertificate.Certs[n].Certificate.PrivateKey and base64 decoded it by hand.

base64 -d <<< THECONTENTOF DomainsCertificate.Certs[n].Certificate.PrivateKey
-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----

(on linux -d, on Mac -D)

.. to a file and referenced the key in gitlab.

registry_nginx['ssl_certificate'] = "/etc/traefik/certs/my.domain.com.pem"
registry_nginx['ssl_certificate_key'] = "/etc/traefik/private/letsencrypt.key"

ldez · 2017-09-22T17:42:02Z

have you tried with the new version ? #2161

frederik · 2017-09-22T18:16:54Z

I have now. This works for me. 👌

poVoq · 2018-03-26T05:02:31Z

Getting an error in line 71 with this

./dumpcerts.sh /opt/traefik/acme.json /opt/traefik/certs/
./dumpcerts.sh: Zeile 71: $1 ist nicht gesetzt.

"ist nicht gesetzt" means "is not set"

torarnv · 2019-01-18T10:35:09Z

Couldn’t this be a built in feature of traefik? Dumping the certs to a volume to be shared live with other software? Why require an external cron-driven export command to be issued, adding complexity for the use-case?

ldez · 2019-01-18T10:36:38Z

@torarnv #1512

brianredbeard mentioned this pull request Apr 23, 2017

ACME file format description #1012

Closed

timoreimann suggested changes Apr 23, 2017

View reviewed changes

ldez added contributor/waiting-for-corrections area/acme labels Apr 23, 2017

ldez added the status/2-needs-review label May 21, 2017

timoreimann force-pushed the dumpcerts branch from 0e8609f to 1a17981 Compare June 22, 2017 22:50

brianredbeard and others added 2 commits June 23, 2017 00:50

Address review comments.

0d5cad5

timoreimann force-pushed the dumpcerts branch from 1a17981 to 0d5cad5 Compare June 22, 2017 22:50

timoreimann approved these changes Jun 22, 2017

View reviewed changes

timoreimann removed the contributor/waiting-for-corrections label Jun 22, 2017

ldez closed this Jun 23, 2017

ldez reopened this Jun 23, 2017

emilevauge approved these changes Jun 24, 2017

View reviewed changes

ldez approved these changes Jun 24, 2017

View reviewed changes

ldez merged commit ceec810 into traefik:master Jun 24, 2017

ldez removed the status/2-needs-review label Jun 24, 2017

ldez modified the milestone: 1.4 Jul 7, 2017

ldez added the kind/enhancement a new or improved feature. label Aug 25, 2017

mathuin mentioned this pull request Apr 1, 2018

contrib: dumpcerts.sh needs updating #3115

Closed

dtomcej mentioned this pull request May 29, 2018

Add an export ACME command #1512

Open

This comment has been minimized.

Sign in to view

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

contrib: Dump keys/certs from acme.json to files #1484

contrib: Dump keys/certs from acme.json to files #1484

brianredbeard commented Apr 23, 2017

timoreimann left a comment •

edited

Loading

timoreimann Apr 23, 2017

timoreimann Apr 23, 2017

timoreimann Apr 23, 2017

brianredbeard Apr 25, 2017

timoreimann Apr 23, 2017

timoreimann Apr 23, 2017

timoreimann Apr 23, 2017

emilevauge Apr 25, 2017

timoreimann Apr 25, 2017 •

edited

Loading

timoreimann Apr 23, 2017

timoreimann Apr 23, 2017

timoreimann Apr 23, 2017

brianredbeard Apr 25, 2017

timoreimann Apr 23, 2017

emilevauge commented Apr 24, 2017

brianredbeard commented Apr 25, 2017

ldez commented Apr 29, 2017

emilevauge commented May 19, 2017

timoreimann commented Jun 22, 2017

brianredbeard commented Jun 23, 2017

ldez commented Jun 23, 2017

emilevauge left a comment

kachkaev commented Jul 1, 2017

frederik commented Sep 2, 2017

jujes commented Sep 22, 2017 •

edited

Loading

kachkaev commented Sep 22, 2017 •

edited

Loading

frederik commented Sep 22, 2017

ldez commented Sep 22, 2017

frederik commented Sep 22, 2017

poVoq commented Mar 26, 2018

torarnv commented Jan 18, 2019

ldez commented Jan 18, 2019

This comment has been minimized.

		}


		acme="${1}"



		acme="${1}"
		certdir="${2}"

contrib: Dump keys/certs from acme.json to files #1484

contrib: Dump keys/certs from acme.json to files #1484

Conversation

brianredbeard commented Apr 23, 2017

timoreimann left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timoreimann Apr 25, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emilevauge commented Apr 24, 2017

brianredbeard commented Apr 25, 2017

ldez commented Apr 29, 2017

emilevauge commented May 19, 2017

timoreimann commented Jun 22, 2017

brianredbeard commented Jun 23, 2017

ldez commented Jun 23, 2017

emilevauge left a comment

Choose a reason for hiding this comment

kachkaev commented Jul 1, 2017

frederik commented Sep 2, 2017

jujes commented Sep 22, 2017 • edited Loading

kachkaev commented Sep 22, 2017 • edited Loading

frederik commented Sep 22, 2017

ldez commented Sep 22, 2017

frederik commented Sep 22, 2017

poVoq commented Mar 26, 2018

torarnv commented Jan 18, 2019

ldez commented Jan 18, 2019

This comment has been minimized.

timoreimann left a comment •

edited

Loading

timoreimann Apr 25, 2017 •

edited

Loading

jujes commented Sep 22, 2017 •

edited

Loading

kachkaev commented Sep 22, 2017 •

edited

Loading