Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updates getPrivateKeyPair to throw exception when private key pair does not exist #18263

Merged
merged 10 commits into from
Oct 21, 2022
Merged

Updates getPrivateKeyPair to throw exception when private key pair does not exist #18263

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
0afe002
Changes method from returning null to throwing Exception to match met…
ryankfu Oct 20, 2022
ea97c72
Added javadoc explaining that KeyPair should not be returning null
ryankfu Oct 20, 2022
16fa02b
Moved ConnectionErrorException to commons package
ryankfu Oct 20, 2022
a60b701
Adds error handling for SshWrappedDestinations and markdown to includ…
ryankfu Oct 21, 2022
d6932e5
Bumps version numbers for Postgres, MySQL, MSSQL
ryankfu Oct 21, 2022
5450f6d
auto-bump connector version
octavia-squidington-iii Oct 21, 2022
76d9ab5
auto-bump connector version
octavia-squidington-iii Oct 21, 2022
bb3620c
Merge branch 'ryan/add-error-messaging-null-private-key-pair' of http…
octavia-squidington-iii Oct 21, 2022
ddb534c
auto-bump connector version
octavia-squidington-iii Oct 21, 2022
bf8d49a
Merge branch 'ryan/add-error-messaging-null-private-key-pair' of http…
octavia-squidington-iii Oct 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...b/exception/ConnectionErrorException.java → .../exceptions/ConnectionErrorException.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
* Copyright (c) 2022 Airbyte, Inc., all rights reserved.
*/

package io.airbyte.db.exception;
package io.airbyte.commons.exceptions;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For context, this class was moved to the commons package so that it can be utilized in the SshTunnel class and with the Db classes


public class ConnectionErrorException extends RuntimeException {

Expand Down
6 changes: 3 additions & 3 deletions airbyte-config/init/src/main/resources/seed/source_definitions.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -644,7 +644,7 @@
- name: Microsoft SQL Server (MSSQL)
sourceDefinitionId: b5ea17b1-f170-46dc-bc31-cc744ca984c1
dockerRepository: airbyte/source-mssql
dockerImageTag: 0.4.22
dockerImageTag: 0.4.23
documentationUrl: https://docs.airbyte.com/integrations/sources/mssql
icon: mssql.svg
sourceType: database
Expand Down Expand Up @@ -692,7 +692,7 @@
- name: MySQL
sourceDefinitionId: 435bb9a5-7887-4809-aa58-28c27df0d7ad
dockerRepository: airbyte/source-mysql
dockerImageTag: 1.0.6
dockerImageTag: 1.0.7
documentationUrl: https://docs.airbyte.com/integrations/sources/mysql
icon: mysql.svg
sourceType: database
Expand Down Expand Up @@ -850,7 +850,7 @@
- name: Postgres
sourceDefinitionId: decd338e-5647-4c0b-adf4-da0e75f5a750
dockerRepository: airbyte/source-postgres
dockerImageTag: 1.0.18
dockerImageTag: 1.0.19
documentationUrl: https://docs.airbyte.com/integrations/sources/postgres
icon: postgresql.svg
sourceType: database
Expand Down
6 changes: 3 additions & 3 deletions airbyte-config/init/src/main/resources/seed/source_specs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6214,7 +6214,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-mssql:0.4.22"
- dockerImage: "airbyte/source-mssql:0.4.23"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/destinations/mssql"
connectionSpecification:
Expand Down Expand Up @@ -7044,7 +7044,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-mysql:1.0.6"
- dockerImage: "airbyte/source-mysql:1.0.7"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/mysql"
connectionSpecification:
Expand Down Expand Up @@ -8676,7 +8676,7 @@
supportsNormalization: false
supportsDBT: false
supported_destination_sync_modes: []
- dockerImage: "airbyte/source-postgres:1.0.18"
- dockerImage: "airbyte/source-postgres:1.0.19"
spec:
documentationUrl: "https://docs.airbyte.com/integrations/sources/postgres"
connectionSpecification:
Expand Down
2 changes: 1 addition & 1 deletion airbyte-db/db-lib/src/main/java/io/airbyte/db/jdbc/DefaultJdbcDatabase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@
package io.airbyte.db.jdbc;

import com.google.errorprone.annotations.MustBeClosed;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.commons.functional.CheckedConsumer;
import io.airbyte.commons.functional.CheckedFunction;
import io.airbyte.db.JdbcCompatibleSourceOperations;
import io.airbyte.db.exception.ConnectionErrorException;
import java.sql.Connection;
import java.sql.DatabaseMetaData;
import java.sql.PreparedStatement;
Expand Down
2 changes: 1 addition & 1 deletion airbyte-db/db-lib/src/main/java/io/airbyte/db/mongodb/MongoDatabase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoCursor;
import com.mongodb.client.MongoIterable;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.commons.functional.CheckedFunction;
import io.airbyte.commons.util.MoreIterators;
import io.airbyte.db.AbstractDatabase;
import io.airbyte.db.exception.ConnectionErrorException;
import java.util.Collections;
import java.util.List;
import java.util.Optional;
Expand Down
6 changes: 4 additions & 2 deletions ...ntegrations/bases/base-java/src/main/java/io/airbyte/integrations/base/ssh/SshTunnel.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

import com.fasterxml.jackson.databind.JsonNode;
import com.google.common.base.Preconditions;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.commons.functional.CheckedConsumer;
import io.airbyte.commons.functional.CheckedFunction;
import io.airbyte.commons.json.Jsons;
Expand Down Expand Up @@ -298,10 +299,11 @@ public void close() {
* From the OPENSSH private key string, use mina-sshd to deserialize the key pair, reconstruct the
* keys from the key info, and return the key pair for use in authentication.
*
* @return The {@link KeyPair} to add - may not be {@code null}
* @see <a href=
* "https://javadoc.io/static/org.apache.sshd/sshd-common/2.8.0/org/apache/sshd/common/config/keys/loader/KeyPairResourceLoader.html#loadKeyPairs-org.apache.sshd.common.session.SessionContext-org.apache.sshd.common.util.io.resource.IoResource-org.apache.sshd.common.config.keys.FilePasswordProvider-">loadKeyPairs()</a>
*/
KeyPair getPrivateKeyPair() throws IOException, GeneralSecurityException {
KeyPair getPrivateKeyPair() throws IOException, GeneralSecurityException, ConnectionErrorException {
final String validatedKey = validateKey();
final var keyPairs = SecurityUtils
.getKeyPairResourceParser()
Expand All @@ -310,7 +312,7 @@ KeyPair getPrivateKeyPair() throws IOException, GeneralSecurityException {
if (keyPairs != null && keyPairs.iterator().hasNext()) {
return keyPairs.iterator().next();
}
return null;
throw new ConnectionErrorException("Unable to load private key pairs, verify key pairs are properly inputted");
}

private String validateKey() {
Expand Down
18 changes: 15 additions & 3 deletions ...bases/base-java/src/main/java/io/airbyte/integrations/base/ssh/SshWrappedDestination.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,20 @@

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.commons.json.Jsons;
import io.airbyte.commons.resources.MoreResources;
import io.airbyte.integrations.base.AirbyteMessageConsumer;
import io.airbyte.integrations.base.AirbyteTraceMessageUtility;
import io.airbyte.integrations.base.Destination;
import io.airbyte.protocol.models.AirbyteConnectionStatus;
import io.airbyte.protocol.models.AirbyteConnectionStatus.Status;
import io.airbyte.protocol.models.AirbyteMessage;
import io.airbyte.protocol.models.ConfiguredAirbyteCatalog;
import io.airbyte.protocol.models.ConnectorSpecification;
import java.util.List;
import java.util.function.Consumer;
import org.apache.sshd.common.SshException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand All @@ -42,7 +46,7 @@ public SshWrappedDestination(final Destination delegate,
}

public SshWrappedDestination(final Destination delegate,
String endPointKey) {
final String endPointKey) {
this.delegate = delegate;
this.endPointKey = endPointKey;
this.portKey = null;
Expand All @@ -60,8 +64,16 @@ public ConnectorSpecification spec() throws Exception {

@Override
public AirbyteConnectionStatus check(final JsonNode config) throws Exception {
return (endPointKey != null) ? SshTunnel.sshWrap(config, endPointKey, delegate::check)
: SshTunnel.sshWrap(config, hostKey, portKey, delegate::check);
try {
return (endPointKey != null) ? SshTunnel.sshWrap(config, endPointKey, delegate::check)
: SshTunnel.sshWrap(config, hostKey, portKey, delegate::check);
} catch (final SshException | ConnectionErrorException e) {
final String sshErrorMessage = "Could not connect with provided SSH configuration. Error: " + e.getMessage();
AirbyteTraceMessageUtility.emitConfigErrorTrace(e, sshErrorMessage);
return new AirbyteConnectionStatus()
.withStatus(Status.FAILED)
.withMessage(sshErrorMessage);
}
}

@Override
Expand Down
3 changes: 2 additions & 1 deletion ...ions/bases/base-java/src/main/java/io/airbyte/integrations/base/ssh/SshWrappedSource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
package io.airbyte.integrations.base.ssh;

import com.fasterxml.jackson.databind.JsonNode;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.commons.util.AutoCloseableIterator;
import io.airbyte.commons.util.AutoCloseableIterators;
import io.airbyte.integrations.base.AirbyteTraceMessageUtility;
Expand Down Expand Up @@ -42,7 +43,7 @@ public ConnectorSpecification spec() throws Exception {
public AirbyteConnectionStatus check(final JsonNode config) throws Exception {
try {
return SshTunnel.sshWrap(config, hostKey, portKey, delegate::check);
} catch (final SshException e) {
} catch (final SshException | ConnectionErrorException e) {
final String sshErrorMessage = "Could not connect with provided SSH configuration. Error: " + e.getMessage();
AirbyteTraceMessageUtility.emitConfigErrorTrace(e, sshErrorMessage);
return new AirbyteConnectionStatus()
Expand Down
2 changes: 1 addition & 1 deletion ...-jdbc/src/main/java/io/airbyte/integrations/destination/jdbc/AbstractJdbcDestination.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
import static io.airbyte.integrations.base.errors.messages.ErrorMessage.getErrorMessage;

import com.fasterxml.jackson.databind.JsonNode;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.commons.map.MoreMaps;
import io.airbyte.db.exception.ConnectionErrorException;
import io.airbyte.db.factory.DataSourceFactory;
import io.airbyte.db.jdbc.DefaultJdbcDatabase;
import io.airbyte.db.jdbc.JdbcDatabase;
Expand Down
2 changes: 1 addition & 1 deletion ...ion-jdbc/src/main/java/io/airbyte/integrations/destination/jdbc/copy/CopyDestination.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import static io.airbyte.integrations.base.errors.messages.ErrorMessage.getErrorMessage;

import com.fasterxml.jackson.databind.JsonNode;
import io.airbyte.db.exception.ConnectionErrorException;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.db.factory.DataSourceFactory;
import io.airbyte.db.jdbc.JdbcDatabase;
import io.airbyte.integrations.BaseConnector;
Expand Down
2 changes: 1 addition & 1 deletion ...mongodb/src/main/java/io/airbyte/integrations/destination/mongodb/MongodbDestination.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
import com.mongodb.MongoSecurityException;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoCursor;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.commons.util.MoreIterators;
import io.airbyte.db.exception.ConnectionErrorException;
import io.airbyte.db.jdbc.JdbcUtils;
import io.airbyte.db.mongodb.MongoDatabase;
import io.airbyte.db.mongodb.MongoUtils.MongoInstanceType;
Expand Down
2 changes: 1 addition & 1 deletion ...ation-mysql/src/main/java/io/airbyte/integrations/destination/mysql/MySQLDestination.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@

import com.fasterxml.jackson.databind.JsonNode;
import com.google.common.collect.ImmutableMap;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.commons.json.Jsons;
import io.airbyte.commons.map.MoreMaps;
import io.airbyte.db.exception.ConnectionErrorException;
import io.airbyte.db.factory.DataSourceFactory;
import io.airbyte.db.factory.DatabaseDriver;
import io.airbyte.db.jdbc.JdbcDatabase;
Expand Down
2 changes: 1 addition & 1 deletion .../main/java/io/airbyte/integrations/destination/redshift/RedshiftStagingS3Destination.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@
import static io.airbyte.integrations.destination.s3.S3DestinationConfig.getS3DestinationConfig;

import com.fasterxml.jackson.databind.JsonNode;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.commons.json.Jsons;
import io.airbyte.db.exception.ConnectionErrorException;
import io.airbyte.db.factory.DataSourceFactory;
import io.airbyte.db.jdbc.DefaultJdbcDatabase;
import io.airbyte.db.jdbc.JdbcDatabase;
Expand Down
2 changes: 1 addition & 1 deletion ...source-mongodb-v2/src/main/java/io.airbyte.integrations.source.mongodb/MongoDbSource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@
import com.mongodb.MongoException;
import com.mongodb.MongoSecurityException;
import com.mongodb.client.MongoCollection;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.commons.functional.CheckedConsumer;
import io.airbyte.commons.json.Jsons;
import io.airbyte.commons.util.AutoCloseableIterator;
import io.airbyte.commons.util.AutoCloseableIterators;
import io.airbyte.db.exception.ConnectionErrorException;
import io.airbyte.db.jdbc.JdbcUtils;
import io.airbyte.db.mongodb.MongoDatabase;
import io.airbyte.db.mongodb.MongoUtils;
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-mssql-strict-encrypt/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ ENV APPLICATION source-mssql-strict-encrypt

COPY --from=build /airbyte /airbyte

LABEL io.airbyte.version=0.4.22
LABEL io.airbyte.version=0.4.23
LABEL io.airbyte.name=airbyte/source-mssql-strict-encrypt
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-mssql/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ ENV APPLICATION source-mssql

COPY --from=build /airbyte /airbyte

LABEL io.airbyte.version=0.4.22
LABEL io.airbyte.version=0.4.23
LABEL io.airbyte.name=airbyte/source-mssql
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-mysql-strict-encrypt/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ ENV APPLICATION source-mysql-strict-encrypt

COPY --from=build /airbyte /airbyte

LABEL io.airbyte.version=1.0.6
LABEL io.airbyte.version=1.0.7

LABEL io.airbyte.name=airbyte/source-mysql-strict-encrypt
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-mysql/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ ENV APPLICATION source-mysql

COPY --from=build /airbyte /airbyte

LABEL io.airbyte.version=1.0.6
LABEL io.airbyte.version=1.0.7

LABEL io.airbyte.name=airbyte/source-mysql
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-postgres-strict-encrypt/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ ENV APPLICATION source-postgres-strict-encrypt

COPY --from=build /airbyte /airbyte

LABEL io.airbyte.version=1.0.18
LABEL io.airbyte.version=1.0.19
LABEL io.airbyte.name=airbyte/source-postgres-strict-encrypt
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-postgres/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ ENV APPLICATION source-postgres

COPY --from=build /airbyte /airbyte

LABEL io.airbyte.version=1.0.18
LABEL io.airbyte.version=1.0.19
LABEL io.airbyte.name=airbyte/source-postgres
2 changes: 1 addition & 1 deletion ...tional-db/src/main/java/io/airbyte/integrations/source/relationaldb/AbstractDbSource.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import com.fasterxml.jackson.databind.JsonNode;
import com.google.common.base.Preconditions;
import com.google.common.collect.Lists;
import io.airbyte.commons.exceptions.ConnectionErrorException;
import io.airbyte.commons.features.EnvVariableFeatureFlags;
import io.airbyte.commons.features.FeatureFlags;
import io.airbyte.commons.functional.CheckedConsumer;
Expand All @@ -21,7 +22,6 @@
import io.airbyte.config.helpers.StateMessageHelper;
import io.airbyte.db.AbstractDatabase;
import io.airbyte.db.IncrementalUtils;
import io.airbyte.db.exception.ConnectionErrorException;
import io.airbyte.db.jdbc.JdbcDatabase;
import io.airbyte.integrations.BaseConnector;
import io.airbyte.integrations.base.AirbyteStreamNameNamespacePair;
Expand Down
Loading