Wings in Docker - "x509: certificate signed by unknown authority" on startup

[Sabinno]

Background (please complete the following information):

• Panel or Daemon: Daemon
• Version of Panel/Daemon: Panel: "canary" (:latest docker image tag); Wings: 1.2.3
• Server's OS: CentOS Linux release 8.3.2011
• Your Computer's OS & Browser: Windows 10 20H2, Microsoft Edge (though this is completely unrelated)
• Traefik version: 2.4.2
• Docker server/client version: 20.10.3 Community
• Panel and daemon are behind Cloudflare

Describe the bug
In my particular configuration -- setting up both the panel and wings in docker, panel and daemon behind Cloudflare (SSL mode: Strict) and the panel alone behind Traefik -- I receive the following error (captured from Portainer)

 INFO: [Feb 15 19:51:18.699] fetching list of servers from API
FATAL: [Feb 15 19:51:18.775] failed to load server configurations error=Get "https://panel.argonaut.network/api/remote/servers?per_page=50": x509: certificate signed by unknown authority

Stacktrace:
Get "https://panel.argonaut.network/api/remote/servers?per_page=50": x509: certificate signed by unknown authority
github.com/pterodactyl/wings/cmd.rootCmdRun
	github.com/pterodactyl/wings/cmd/root.go:192
github.com/spf13/cobra.(*Command).execute
	github.com/spf13/[email protected]/command.go:854
github.com/spf13/cobra.(*Command).ExecuteC
	github.com/spf13/[email protected]/command.go:958
github.com/spf13/cobra.(*Command).Execute
	github.com/spf13/[email protected]/command.go:895
github.com/pterodactyl/wings/cmd.Execute
	github.com/pterodactyl/wings/cmd/root.go:61
main.main
	command-line-arguments/wings.go:8
runtime.main
	runtime/proc.go:204
runtime.goexit
	runtime/asm_amd64.s:1374

I have attempted to resolve the issue with the --ignore-certificate-errors startup arg, but receive a different error altogether.

Lastly, I have attempted to resolve this issue, per the advice of Dr3nz4r in #wings-in-docker in Discord, by starting wings with the arg --auto-tls --tls-hostname node1.argonaut.network. This results in the former error log output, verbatim.

I have confirmed that the correct cert/key pair are located in /etc/letsencrypt/live/<host>.
I have confirmed that SELinux has not thrown any errors, and this still occurs in permissive mode.
I have confirmed that, to my knowledge (I have no idea how to tell, but was asked to verify this as a troubleshooting step), the files that should be located in /etc/ssl/certs are present.
I have confirmed that the necessary ports are open.

Below, you may find my configuration files:

• panel docker-compose.yml: https://pastebin.com/fnpAj1Jp
• wings docker-compose.yml: https://pastebin.com/k9QqSgGi
• /etc/pterodactyl/config.yml: https://pastebin.com/yFQ9qkve

To Reproduce
Steps to reproduce the behavior:

1. Install panel and wings in docker with docker-compose files created as-is above.
2. Start wings with docker-compose up -d

Expected behavior
Wings starts normally.

[TDogVoid]

I had this exact issue. This solved it for me
#2961 (comment)

[schrej]

Are your ca ca certificates up to date? The package should be ca-certificates.

[Sabinno]

I had this exact issue. This solved it for me
#2961 (comment)

Check my Wings docker-compose. That's definitely something I made sure I did.

[Sabinno]

Are your ca ca certificates up to date? The package should be ca-certificates.

I will check when I'm at a computer.

[DaneEveritt]

Cannot reproduce, and from what I've seen in the past this has always been an issue with the system ca store not being updated correctly.

[TheFrisianClause]

How did you resolve this, as I currently followed the Wings installation instructions and imported the Letsencrypt certificates generated by ACME in PFsense. But I am getting this error:

FATAL: [Sep 1 21:01:03.689] failed to load server configurations error=http: request creation failed: Get "https://{SERVER_URL}/api/remote/servers?page=0&per_page=50": x509: certificate signed by unknown authority

Stacktrace:
Get "https://{SERVER_URL}/api/remote/servers?page=0&per_page=50": x509: certificate signed by unknown authority
http: request creation failed

I have done some troubleshooting, but cannot resolve the issue...

[schrej]

Try updating your ca-certificates.

[haugli92]

If you moved your panel from another server, renew certificates. That was my problem.

[dylif]

I'm currently having this issue while running both Panel and Wings under docker. My system CA store was correctly mapped for wings to use. Not really sure what could be causing this as running Wings outside of docker fixes this issue.

[Sabinno]

The only way to fix this at this time has been to use --ignore-certificate-errors while running Wings.

[Alth3r]

I have encountered this issue as well and after some detective work I have found the error and a possible solution. Although I am not sure if my way of solving this can be considered "best practice".

Running
docker inspect ghcr.io/pterodactyl/wings
shows that apparently the following file is needed in order to verify a TLS certificate:
"SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"

This file is being mounted via the docker-compose.yml.
So let's take a look at the ca-certificates.crt file on the host system:
ls -la /etc/ssl/certs/ca-certificates.crt
Output:
lrwxrwxrwx 1 root root 49 Sep 5 23:59 /etc/ssl/certs/ca-certificates.crt -> ../../ca-certificates/extracted/tls-ca-bundle.pem

So the ca-certificates.crt is just a symlink which points to a file at the location /etc/ca-certificates/extracted/tls-ca-bundle.pem. This directory is not mounted to the container.
Then I simply tried adding the following line to the volumes section:
- "/etc/ca-certificates/extracted/tls-ca-bundle.pem:/etc/ca-certificates/extracted/tls-ca-bundle.pem"

And voilà the container starts without any error messages. TLS certificates are now being correctly verified.

I hope this helps anyone who might run into the same problem. Have a great day :D

[argonaut-network]

Should be able to work around this with pterodactyl/wings#154

Basically, just mount /etc/ssl/certs/certificates.crt to /etc/pki/ca-trust/extracted/openssl/ca-trust.bundle.crt which is the location of openSSL CA certs on EL/Fedora based distros.

[jonxor]

For future generations that land here, this can happen if you decide to use just the hostname.cer instead of fullchain.cer. Make sure you use fullchain on both wings SSL cert and the panel's SSL cert.