Fix: log error when SSL context building fails

[kares]

There's several changes to make sure plugin provides better user-experience when it comes to logging.
The main motivator for the change set are key issues such as java.lang.IllegalArgumentException: File does not contain valid private key which are only logged upstream at a debug level (see #400).

On LS 7.8 / 7.9 the output when key has invalid format (or a ssl_xxx related configuration issue looks like):
(one needs to use --log.level debug to find out what the issue was)

[2020-11-27T15:40:32,191][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>0.52}
[2020-11-27T15:40:32,211][INFO ][logstash.inputs.beats    ][main] Starting input listener {:address=>"0.0.0.0:5044"}
[2020-11-27T15:40:32,308][ERROR][logstash.inputs.beats    ][main] SSL configuration invalid {:message=>"File does not contain valid private key: /home/kares/workspace/work/elastic/plugins/logstash-input-beats/cert-bundle/logstash.key"}
[2020-11-27T15:40:32,485][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2020-11-27T15:40:32,652][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2020-11-27T15:40:37,717][INFO ][logstash.runner          ] Logstash shut down.
[2020-11-27T15:40:37,730][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

Thus to improve the logger experience there's 3 set of changes:

• log when plugin's SSL context builder fails (user can easily spot key format errors)
• make sure we log configuration errors before raising them (due logstash fails to log configuration error message elastic/logstash#12484)
• on SSL hand-shake failures extract the real-cause as an attempt to log a better info message e.g.

[2020-11-30T12:53:38,997][INFO ][org.logstash.beats.BeatsHandler][main][beats-tls] [local: 0.0.0.0:5044, remote: 192.168.122.59:57466] Handling exception: io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: General SSLEngine problem (caused by: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors)

resolves #400

[robbavey]

Thanks for doing this! Just a few minor comments to clarify/improve consistency of some of the logging statements

[robbavey]

Maybe could be reworded a little?

Suggested change

	configuration_error "Missing required setting ssl_key => ... (due ssl => true)"
	configuration_error "ssl_key => is a required setting when ssl=>true is configured"

[kares]

thanks, fine by me - what I was trying is to be a bit similar to the generic (required) parameter validation:

Missing a required setting for the beats input plugin:

  input {
    beats {
      port => # SETTING MISSING
      ...
    }
  }

[robbavey]

Backtrace useful here?

[kares]

these are plugin 'owned' exceptions but it won't hurt to have a trace with debug logging

[robbavey]

Could also use a slow "pointer"/fast "pointer" to detect exception loops, if an exception loop is common for this kind of error, but this is ok too if not

[kares]

have checked this manually enforcing a TLS certificate validation failure - which gets us 4 levels of nested causes.
at this point I am not sure loops are common just wanted to make sure we do not accidentally end up looping too much here, esp. as these are considered non-fatal and won't halt plugin execution.

imho, what's important is that we try the extraction since the info logging the plugin did previously was not providing useful information on the 'real' cause + also gave 'unclear' logging based on the following condition:

cause.getMessage() == null ? cause.getClass().toString() : cause.getMessage()
which made me previously believe we had a javax.net.ssl.SSLHandshakeException: General SSLEngine problem around here while in fact it was wrapped in a io.netty.handler.codec.DecoderException with "javax.net.ssl.SSLHandshakeException: General SSLEngine problem" as the message.

the whole exception stack is later printed by Netty's logger.

[robbavey]

Makes sense - I only asked about Exception loops because I encountered them working on the JMS plugin, and added a guard against them.

[robbavey]

LGTM, assuming travis (eventually) is green