Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

restrict TLS cipher suites of the server #5245

Merged
merged 3 commits into from
Jan 13, 2018
Merged

Conversation

aead
Copy link
Member

@aead aead commented Nov 28, 2017

Description

This change restircts the supported cipher suites of the minio server.
The server only supports AEAD ciphers (Chacha20Poly1305 and AES-GCM)

The supported cipher suites are:

  • tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
  • tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
  • tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  • tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  • tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  • tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384

Motivation and Context

Fixes #5244
Fixes #5291

How Has This Been Tested?

manually

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added unit tests to cover my changes.
  • I have added/updated functional tests in mint. (If yes, add mint PR # here: )
  • All new and existing tests passed.

@@ -155,8 +155,18 @@ func NewServer(addrs []string, handler http.Handler, certificate *tls.Certificat
if certificate != nil {
tlsConfig = &tls.Config{
PreferServerCipherSuites: true,
MinVersion: tls.VersionTLS12,
NextProtos: []string{"http/1.1", "h2"},
CipherSuites: []uint16{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should take this list as perhaps an argument to NewServer() function, in future we might need to change these as well. We can take this as an input from top level and not default/restrict this package to the following ciphers.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree 👍

@codecov
Copy link

codecov bot commented Nov 28, 2017

Codecov Report

Merging #5245 into master will increase coverage by 0.3%.
The diff coverage is 90%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #5245     +/-   ##
=========================================
+ Coverage   61.12%   61.43%   +0.3%     
=========================================
  Files         201      201             
  Lines       29509    29303    -206     
=========================================
- Hits        18038    18002     -36     
+ Misses      10083     9908    -175     
- Partials     1388     1393      +5
Impacted Files Coverage Δ
pkg/http/server.go 84.81% <90%> (+58.78%) ⬆️
cmd/xl-v1-list-objects-heal.go 71.08% <0%> (-3.07%) ⬇️
cmd/admin-handlers.go 53.49% <0%> (-2.22%) ⬇️
cmd/xl-v1-healing-common.go 78.02% <0%> (-1.65%) ⬇️
cmd/object-api-errors.go 62.26% <0%> (-1.26%) ⬇️
cmd/jwt.go 75% <0%> (-1.12%) ⬇️
cmd/bucket-policy-handlers.go 88.18% <0%> (-0.22%) ⬇️
cmd/bucket-policy.go 62.13% <0%> (-0.08%) ⬇️
cmd/bucket-handlers.go 69.09% <0%> (-0.08%) ⬇️
cmd/xl-v1-object.go 79.05% <0%> (-0.05%) ⬇️
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3f8379d...ce07746. Read the comment docs.

@harshavardhana
Copy link
Member

Go implementation of CBC does not contain effective countermeasures
against timing attacks.

Some counter measures implemented

golang/go@f28cf83
golang/go@4822e76
golang/go@48d8edb

Refer here golang/go#13385

Disabling CBC for 128bit and above can have real issues with clients like CloudBerry which was using CBC. Historically we had this in the past

https://github.com/minio/minio/pull/3608/files#diff-e6845802830d0358dfc98e01152842c2R379

Thus we had to bring back CBC to support it, so we should validate those clients before accepting this patch and also have a thorough review such that we are not too strict about this.

@aead
Copy link
Member Author

aead commented Nov 28, 2017

Oh okay, wasn't aware that Filippo fixed some CBC timing issues. So we can "safely" enable CBC-*-SHA suites. However SHA256 suites are still vulnerable.

Agree that we have to check clients.

Copy link
Member

@harshavardhana harshavardhana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM tested

@deekoder deekoder requested review from donatello and harshavardhana and removed request for balamurugana December 12, 2017 18:48
harshavardhana
harshavardhana previously approved these changes Dec 12, 2017
@aead
Copy link
Member Author

aead commented Dec 12, 2017

Disabling RSA PKCS v1.5 ciphers seems to make no difference. However it's theoretically possible that there are clients which does not support ECC... As far as I can see disabling RSA-only ciphers doesn't break anything I'm aware of...

harshavardhana
harshavardhana previously approved these changes Dec 12, 2017
Copy link
Member

@harshavardhana harshavardhana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will perhaps tell the clients to upgrade rather use vulnerable access. LGTM

cmd/globals.go Outdated
@@ -167,6 +167,16 @@ var (
colorYellow = color.New(color.FgYellow).SprintfFunc()
)

// Secure Go implementations of TLS ciphers
var tlsCipherSuites = []uint16{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is more secure TLS configuration, how about making this default cipher suite for http server in case of TLS enabled? You always has an option to override this default behavior.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't understand - what do you mean with "making this the default cipher suites". These ciphers are used as the "default" cipher suites for the server and the gateway. Or do I miss something?
See https://github.com/minio/minio/pull/5245/files#diff-736572b438aa05056f334265beb9507bR158

Copy link
Member

@balamurugana balamurugana Dec 13, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My suggestion is below

  1. move tlsCipherSuites to pkg/http/server.go itself i.e. no definition in cmd/*.go
  2. there is no cipherSuites argument to NewServer() i.e. tlsCipherSuites is set always for TLS.
  3. if user wants to override this, he could directly set Server.TLSConfig accordingly to his need.
  4. this way there is no change in cmd/*.go

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's exactly how the patch looked before but harsha mentioned:

We should take this list as perhaps an argument to NewServer() function, in future we might need to change these as well. We can take this as an input from top level and not default/restrict this package to the following ciphers.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harshavardhana As the approach provides support to the use case you mentioned, do you have any other concern to do the change?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That won't be a problem, since the TLSConfig can be overriden. I guess having a default is fine for now.

donatello
donatello previously approved these changes Dec 13, 2017
@aead
Copy link
Member Author

aead commented Dec 13, 2017

Okay, reverted changes according to @balamurugana comment.
@harshavardhana @donatello PTAL

Copy link
Member

@harshavardhana harshavardhana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GIt commit message here still says we are allowing following ciphers but code is changed can you amend your commit as well @aead ?

 - tls.TLS_RSA_WITH_AES_128_GCM_SHA256
 - tls.TLS_RSA_WITH_AES_256_GCM_SHA384

Copy link
Member

@harshavardhana harshavardhana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A typo s/restircts/restricts/

because the Go implementation of CBC does not contain effective countermeasures against timing attacks.

Do we still need to mention this? @aead

Further disable RSA PKCSv1.5 cipher suites because of timing issues as shown with the ROBOT attack.

This sentence is not needed perhaps. Unless you want to really say why we even removed RSA_AES support.

// NewServer - creates new HTTP server using given arguments.
func NewServer(addrs []string, handler http.Handler, certificate *tls.Certificate) *Server {
var tlsConfig *tls.Config
if certificate != nil {
tlsConfig = &tls.Config{
PreferServerCipherSuites: true,
CipherSuites: defaultCipherSuites,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add a unit tests to address below cases for default http server and TLSConfig'ed http server?

  1. http client connecting with defaultCipherSuites
  2. http client connecting without defaultCipherSuites
  3. http client connecting with unsupported cipher

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added tests - PTAL

@aead
Copy link
Member Author

aead commented Dec 14, 2017

@harshavardhana
Updated commit msg, thanks.
I also reduced commit message - However added comments so it is understandable why we disabled a specific cipher.

BTW: I disabled RSA-PKCS1-v1.5 not because of ROBOT itself (Go stack is not vulnerable to ROBOT)
but as the name says (ROBOT = Return Of Bleichenbacher's Oracle Threat) its again a variant of the Bleichenbacher timing attack. See this -> It's not unlikely that it will strike again...

@aead aead force-pushed the tls-ciphersuite branch 2 times, most recently from 7da2406 to 9940668 Compare December 14, 2017 22:17
@balamurugana
Copy link
Member

@aead You go ahead and remove unit tests added in this PR. I will send a separate PR to address unit tests.

@aead
Copy link
Member Author

aead commented Jan 10, 2018

@balamurugana Have added tests - Can you take a look and if something is missing from your point of view can you address?

@balamurugana
Copy link
Member

@aead My previous comment You could have one test function doing multiple test cases i.e.func TestServerStart() we would need. is not yet addressed.

Let me know I can send a PR with the fix to your fork

@balamurugana
Copy link
Member

@aead Please review aead#3

@aead
Copy link
Member Author

aead commented Jan 12, 2018

@harshavardhana @donatello
PTAL - Merged requested changes by @balamurugana

@balamurugana
Copy link
Member

Mint Automation

Test Result
mint-xl.sh more...
mint-fs.sh more...
mint-dist-xl.sh more...
mint-gateway-s3.sh more...
mint-gateway-azure.sh more...

5245-ce07746/mint-gateway-azure.sh.log:

Running with
SERVER_ENDPOINT: minikube:31111
ACCESS_KEY:      minioazure1
SECRET_KEY:      ***REDACTED***
ENABLE_HTTPS:    0
SERVER_REGION:   us-east-1
MINT_DATA_DIR:   /mint/data
MINT_MODE:       core

To get logs, run 'docker cp a6e36aab9e03:/mint/log /tmp/mint-logs'
(1/11) Running aws-sdk-go tests ... done in 1 seconds
(2/11) Running aws-sdk-php tests ... done in 1 minutes and 21 seconds
(3/11) Running aws-sdk-ruby tests ... done in 1 minutes and 5 seconds
(4/11) Running awscli tests ... done in 9 minutes and 5 seconds
(5/11) Running mc tests ... done in 9 minutes and 30 seconds
(6/11) Running minio-dotnet tests ... FAILED in 3 minutes and 13 seconds

Unhandled Exception: System.AggregateException: One or more errors occurred. (Assert.Fail failed. ) ---> Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.Fail failed. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail()
   at Minio.Functional.Tests.FunctionalTest.<PutObject_Task>d__52.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 755
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Minio.Functional.Tests.FunctionalTest.<ListObjects_Test5>d__72.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 1633
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Minio.Functional.Tests.FunctionalTest.Main(String[] args) in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 216

Executed 5 out of 11 tests successfully.

5245-ce07746/mint-fs.sh.log:

Running with
SERVER_ENDPOINT: minikube:30539
ACCESS_KEY:      minio
SECRET_KEY:      ***REDACTED***
ENABLE_HTTPS:    0
SERVER_REGION:   us-east-1
MINT_DATA_DIR:   /mint/data
MINT_MODE:       core

To get logs, run 'docker cp 12c36d82c300:/mint/log /tmp/mint-logs'
(1/11) Running aws-sdk-go tests ... done in 0 seconds
(2/11) Running aws-sdk-php tests ... done in 50 seconds
(3/11) Running aws-sdk-ruby tests ... done in 6 seconds
(4/11) Running awscli tests ... done in 6 minutes and 47 seconds
(5/11) Running mc tests ... done in 2 minutes and 43 seconds
(6/11) Running minio-dotnet tests ... FAILED in 1 minutes and 59 seconds

Unhandled Exception: System.AggregateException: One or more errors occurred. (Assert.Fail failed. ) ---> Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.Fail failed. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail()
   at Minio.Functional.Tests.FunctionalTest.<PutObject_Task>d__52.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 755
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Minio.Functional.Tests.FunctionalTest.<ListObjects_Test5>d__72.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 1633
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Minio.Functional.Tests.FunctionalTest.Main(String[] args) in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 216

Executed 5 out of 11 tests successfully.

5245-ce07746/mint-xl.sh.log:

Running with
SERVER_ENDPOINT: minikube:31839
ACCESS_KEY:      minio
SECRET_KEY:      ***REDACTED***
ENABLE_HTTPS:    0
SERVER_REGION:   us-east-1
MINT_DATA_DIR:   /mint/data
MINT_MODE:       core

To get logs, run 'docker cp 83a5ef885f38:/mint/log /tmp/mint-logs'
(1/11) Running aws-sdk-go tests ... done in 0 seconds
(2/11) Running aws-sdk-php tests ... done in 1 minutes and 5 seconds
(3/11) Running aws-sdk-ruby tests ... done in 15 seconds
(4/11) Running awscli tests ... done in 6 minutes and 57 seconds
(5/11) Running mc tests ... done in 3 minutes and 45 seconds
(6/11) Running minio-dotnet tests ... FAILED in 3 minutes and 18 seconds

Unhandled Exception: System.AggregateException: One or more errors occurred. (Assert.Fail failed. ) ---> Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.Fail failed. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail()
   at Minio.Functional.Tests.FunctionalTest.<PutObject_Task>d__52.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 755
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Minio.Functional.Tests.FunctionalTest.<ListObjects_Test5>d__72.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 1633
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Minio.Functional.Tests.FunctionalTest.Main(String[] args) in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 216

Executed 5 out of 11 tests successfully.

5245-ce07746/mint-gateway-s3.sh.log:

Running with
SERVER_ENDPOINT: minikube:31642
ACCESS_KEY:      AKIAJH7GQKYNU3C2ADXA
SECRET_KEY:      ***REDACTED***
ENABLE_HTTPS:    0
SERVER_REGION:   us-east-1
MINT_DATA_DIR:   /mint/data
MINT_MODE:       core

To get logs, run 'docker cp 72314a057327:/mint/log /tmp/mint-logs'
(1/11) Running aws-sdk-go tests ... done in 2 seconds
(2/11) Running aws-sdk-php tests ... done in 1 minutes and 28 seconds
(3/11) Running aws-sdk-ruby tests ... done in 1 minutes and 31 seconds
(4/11) Running awscli tests ... done in 8 minutes and 52 seconds
(5/11) Running mc tests ... done in 9 minutes and 23 seconds
(6/11) Running minio-dotnet tests ... FAILED in 2 minutes and 23 seconds

Unhandled Exception: System.AggregateException: One or more errors occurred. (Assert.Fail failed. ) ---> Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.Fail failed. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail()
   at Minio.Functional.Tests.FunctionalTest.<PutObject_Task>d__52.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 755
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Minio.Functional.Tests.FunctionalTest.<ListObjects_Test5>d__72.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 1633
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Minio.Functional.Tests.FunctionalTest.Main(String[] args) in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 216

Executed 5 out of 11 tests successfully.

5245-ce07746/mint-dist-xl.sh.log:

Running with
SERVER_ENDPOINT: minikube:30706
ACCESS_KEY:      minio
SECRET_KEY:      ***REDACTED***
ENABLE_HTTPS:    0
SERVER_REGION:   us-east-1
MINT_DATA_DIR:   /mint/data
MINT_MODE:       core

To get logs, run 'docker cp 6a4c2284170a:/mint/log /tmp/mint-logs'
(1/11) Running aws-sdk-go tests ... done in 3 seconds
(2/11) Running aws-sdk-php tests ... done in 1 minutes and 8 seconds
(3/11) Running aws-sdk-ruby tests ... done in 1 minutes and 6 seconds
(4/11) Running awscli tests ... done in 7 minutes and 32 seconds
(5/11) Running mc tests ... done in 3 minutes and 17 seconds
(6/11) Running minio-dotnet tests ... FAILED in 4 minutes and 9 seconds

Unhandled Exception: System.AggregateException: One or more errors occurred. (Assert.Fail failed. ) ---> Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.Fail failed. 
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail()
   at Minio.Functional.Tests.FunctionalTest.<PutObject_Task>d__52.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 755
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Minio.Functional.Tests.FunctionalTest.<ListObjects_Test5>d__72.MoveNext() in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 1633
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Minio.Functional.Tests.FunctionalTest.Main(String[] args) in /mint/run/core/minio-dotnet/FunctionalTest.cs:line 216

Executed 5 out of 11 tests successfully.

Copy link
Member

@harshavardhana harshavardhana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@balamurugana balamurugana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nitisht nitisht merged commit dd202a1 into minio:master Jan 13, 2018
@gowithplanb
Copy link

This patch has broken CloudBerry support for Minio since CloudBerry clients are now unable to initialise a TLS connection.

[2018-02-13T10:25:09.295333193Z] [ERROR] TLS handshake failed with new connection CLIENTIP:PORT at server SERVERIP:PORT (tls: no cipher suite supported by both client and server)
Trace: 1: /q/.q/sources/gopath/src/github.com/minio/minio/pkg/http/listener.go:177:http.(*httpListener).start.func2()

@aead
Copy link
Member Author

aead commented Feb 13, 2018

@gowithplanb
What product/version of cloudberry do you use? Can you provide the cipher suites which are tried during the handshake? / A recorded TLS handshake?

@gowithplanb
Copy link

gowithplanb commented Feb 13, 2018

I'm using the latest stable version 5.8 for MBS customers (similar to the one publically available). I've captured traffic between client and server, and this is what the client offers:

Secure Sockets Layer
TLSv1.2 Record Layer: Handshake Protocol: Client Hello
Content Type: Handshake (22)
Version: TLS 1.2 (0x0303)
Length: 179
Handshake Protocol: Client Hello
Handshake Type: Client Hello (1)
Length: 175
Version: TLS 1.2 (0x0303)
Random: 5a82eb24e65476fd97df8b7c48fbba021bc694441ccc9f1f...
Session ID Length: 0
Cipher Suites Length: 56
Cipher Suites (28 suites)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
Cipher Suite: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x009f)
Cipher Suite: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)
Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA256 (0x003d)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA256 (0x003c)
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 (0x006a)
Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 (0x0040)
Cipher Suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA (0x0038)
Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA (0x0032)
Cipher Suite: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)
Cipher Suite: TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA (0x0013)
Cipher Suite: TLS_RSA_WITH_RC4_128_SHA (0x0005)
Cipher Suite: TLS_RSA_WITH_RC4_128_MD5 (0x0004)
Compression Methods Length: 1
Compression Methods (1 method)
Compression Method: null (0)
Extensions Length: 78
Extension: server_name (len=25)
Extension: supported_groups (len=6)
Extension: ec_point_formats (len=2)
Extension: signature_algorithms (len=20)
Extension: extended_master_secret (len=0)
Extension: renegotiation_info (len=1)

Minio then responds with:

Secure Sockets Layer
TLSv1.2 Record Layer: Alert (Level: Fatal, Description: Handshake Failure)
Content Type: Alert (21)
Version: TLS 1.2 (0x0303)
Length: 2
Alert Message
Level: Fatal (2)
Description: Handshake Failure (40)

@aead
Copy link
Member Author

aead commented Feb 13, 2018

Thanks @gowithplanb I guess you are using a RSA certificate and the client does not support ECDHE_RSA with AES_GCM or CHACHA20_POLY1305. For a ECDSA certificate the client and server should be able to negotiate a cipher suite. (AES_GCM).

The problem is that the Go TLS stack only contains timing attack countermeasures for TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA and TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA and those are not perfect. So we disabled them for security reasons. I'll come back to you with more details on how that issue can be solved.

@gowithplanb
Copy link

@aead I have contact with CloudBerry about this too now. They're informed about it and are looking into it as well. I will not upgrade Minio in production until this is resolved.

@aead
Copy link
Member Author

aead commented Feb 13, 2018

@gowithplanb

I have contact with CloudBerry about this too now.

Great, this is actually a Cloudberry configuration issue. The correct way to fix that is:

  • Cloudberry should enable AES_GCM for ECDHE_RSA cipher suites. They should be able to do that since they already support AES_GCM for ECDHE_ECDSA.
  • Further this issue is MBS software specific. CBB desktop client doesn't have this problem.

You can point them also to this PR thread so they can reach out if the need more information.

@gowithplanb
Copy link

Thanks @aead! PR pointed out to them.

@aead aead deleted the tls-ciphersuite branch March 5, 2018 22:54
@topeju
Copy link

topeju commented Mar 7, 2018

Is it just me or does this PR also prevent Windows 8.1 / Windows Server 2012 R2 from connecting to Minio? IE on Windows Server 2012 R2 just reports

Turn on TLS 1.0, TLS 1.1, and TLS 1.2 in Advanced settings and try connecting to [https URL to the minio server] again. If this error persists, it is possible that this site uses an unsupported protocol or cipher suite such as RC4 (link for the details), which is not considered secure

when all of these TLS versions are enabled in IE.

@aead
Copy link
Member Author

aead commented Mar 7, 2018

@topeju
Just from looking at the a mircosoft blog post they say:

TLS 1.1 & TLS 1.2 are enabled by default on post Windows 8.1 releases. Prior to that they were disabled by default. So the administrators have to enable the settings manually via the registry.

So my guess from the reported msg would be that TLS 1.2 is not enabled on your Windows 8.1 / Windows Server 2012 R2 machine. Further I don't think that this PR causes the problem because according to the ms doc the ms TLS 1.2 stack supports the named cipher suites.

So can you try to enable TLS 1.2 and check if the issue still exists? (on all participating machines)

@topeju
Copy link

topeju commented Mar 7, 2018

No, TLS 1.2 is (and was) already enabled.

When I check it with cipherscan, Minio seems to report supporting ECDHE-RSA-CHACHA20-POLY1305, ECDHE-RSA-AES128-GCM-SHA256, and ECDHE-RSA-AES256-GCM-SHA384. In the Microsoft documentation (which you linked to), no ChaCha ciphers are listed, so the first one is out. All the ECDHE_RSA cipher suites listed seem to use CBC instead of GCM, as far as I can tell, so they too should be incompatible?

@aead
Copy link
Member Author

aead commented Mar 7, 2018

The linked ms doc lists:

  • TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256
  • TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384

However it seems Windows 8.1 does not support AES-GCM with ECDHE_RSA. Since your cipherscan reports the RSA cipher suites I assume you're using an RSA private-key / certificate.
So I think the combination of RSA private-key / certificate and no AES_GCM support for ECDHE_RSA cipher by Windows 8.1 causes this issue.
I will report this internally - A work-around would be to use ECDSA certificates instead of RSA.

BTW: You were right - this is exactly the same issue reported using Cloudberry above.

@topeju
Copy link

topeju commented Mar 7, 2018

Yes, I'm using an RSA private key. I tried to generate an ECDSA key using openssl ecparam -genkey -name prime256v1 -out private.key (from the documentation page) but Minio wouldn't accept the key afterwards, reporting "Invalid SSL certificate file (TLS: private key contains additional data)".

@aead
Copy link
Member Author

aead commented Mar 7, 2018

Oh you are right - I'm sorry about that. I will fix the documentation. In the meantime you can use the following openssl command: openssl ecparam -genkey -name prime256v1 | openssl ec -out private.key

@mhjartstrom
Copy link

OK, so I just wasted a couple of days trying to get a Let's Encrypt ECC key generated using acme.sh to work with Minio, but I kept getting the message "ERROR Unable to load the TLS configuration: The private key contains additional data. Please check your certificate.".

Turns out that for some reason I need to translate the private key by adapting the latter part of the call described above, i.e. if I call "openssl -in .acme.sh/domain/private.key -out .minio/certs/private.key" it'll work.

Now, is there any logic behind the fact that I can use a 4096 bit RSA key straight off, but I need to do some kind of translation in order to use a prime256v1 ECC key? What am I missing here...?

@harshavardhana
Copy link
Member

@mhjartstrom I am opening a new issue with your comment, please add additional details there. Also in future open new issues rather can opening older threads. It makes it easier for us to get to the bottom of this with more details and context.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
9 participants