Skip to content

Commit

Permalink
GODRIVER-1855 Support AWS authentication with temporary credentials i…
Browse files Browse the repository at this point in the history
…n CSFLE (#580)
  • Loading branch information
benjirewis authored and Benjamin Rewis committed Mar 5, 2021
1 parent 66d9882 commit 2c5b75b
Show file tree
Hide file tree
Showing 6 changed files with 401 additions and 13 deletions.
21 changes: 20 additions & 1 deletion .evergreen/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@ functions:
- command: expansions.update
params:
file: mo-expansion.yml

cleanup:
- command: shell.exec
params:
Expand Down Expand Up @@ -292,6 +292,21 @@ functions:
fi
fi
# Set temp credentials for AWS if python3 is available.
#
# Using python3-venv in Ubuntu 14.04 (an OS required for legacy server version
# tasks) requires the use of apt-get, which we wish to avoid. So, we do not set
# a python3 binary on Ubuntu 14.04. Setting AWS temp credentials for legacy
# server version tasks is unneccesary, as temp credentials are only needed on 4.2+.
if [ ! -z ${PYTHON3_BINARY} ]; then
export AWS_ACCESS_KEY_ID="${cse_aws_access_key_id}"
export AWS_SECRET_ACCESS_KEY="${cse_aws_secret_access_key}"
export AWS_DEFAULT_REGION="us-east-1"
${PYTHON3_BINARY} -m venv ./venv
./venv/${VENV_BIN_DIR|bin}/pip3 install boto3
. ${DRIVERS_TOOLS}/.evergreen/csfle/set-temp-creds.sh
fi
export GOFLAGS=-mod=vendor
set +o xtrace
AUTH=${AUTH} \
Expand All @@ -302,6 +317,10 @@ functions:
BUILD_TAGS="-tags cse" \
AWS_ACCESS_KEY_ID="${cse_aws_access_key_id}" \
AWS_SECRET_ACCESS_KEY="${cse_aws_secret_access_key}" \
AWS_DEFAULT_REGION="us-east-1" \
CSFLE_AWS_TEMP_ACCESS_KEY_ID="$CSFLE_AWS_TEMP_ACCESS_KEY_ID" \
CSFLE_AWS_TEMP_SECRET_ACCESS_KEY="$CSFLE_AWS_TEMP_SECRET_ACCESS_KEY" \
CSFLE_AWS_TEMP_SESSION_TOKEN="$CSFLE_AWS_TEMP_SESSION_TOKEN" \
AZURE_TENANT_ID="${cse_azure_tenant_id}" \
AZURE_CLIENT_ID="${cse_azure_client_id}" \
AZURE_CLIENT_SECRET="${cse_azure_client_secret}" \
Expand Down
62 changes: 57 additions & 5 deletions data/client-side-encryption/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ Then for each element in ``tests``:
#. Create a MongoClient.

#. Create a collection object from the MongoClient, using the ``database_name``
and ``collection_name`` fields from the YAML file. Drop the collection
and ``collection_name`` fields from the YAML file. Drop the collection
with writeConcern "majority". If a ``json_schema`` is defined in the test,
use the ``createCollection`` command to explicitly create the collection:

Expand All @@ -147,8 +147,60 @@ Then for each element in ``tests``:

#. Create a **new** MongoClient using ``clientOptions``.

#. If ``autoEncryptOpts`` includes ``aws``, ``azure``, and/or ``gcp`` as a KMS provider, pass in credentials from the environment.
#. If ``autoEncryptOpts`` does not include ``keyVaultNamespace``, default it to ``keyvault.datakeys``.
#. If ``autoEncryptOpts`` includes ``aws``, ``awsTemporary``, ``awsTemporaryNoSessionToken``,
``azure``, and/or ``gcp`` as a KMS provider, pass in credentials from the environment.

- ``awsTemporary``, and ``awsTemporaryNoSessionToken`` require temporary
AWS credentials. These can be retrieved using the csfle `set-temp-creds.sh
<https://github.com/mongodb-labs/drivers-evergreen-tools/tree/master/.evergreen/csfle>`_
script.

- ``aws``, ``awsTemporary``, and ``awsTemporaryNoSessionToken`` are
mutually exclusive.

``aws`` should be substituted with:

.. code:: javascript
"aws": {
"accessKeyId": <set from environment>,
"secretAccessKey": <set from environment>
}
``awsTemporary`` should be substituted with:

.. code:: javascript
"aws": {
"accessKeyId": <set from environment>,
"secretAccessKey": <set from environment>
"sessionToken": <set from environment>
}
``awsTemporaryNoSessionToken`` should be substituted with:

.. code:: javascript
"aws": {
"accessKeyId": <set from environment>,
"secretAccessKey": <set from environment>
}
``gcp`` should be substituted with:

.. code:: javascript
"gcp": {
"email": <set from environment>,
"privateKey": <set from environment>,
}
``azure`` should be substituted with:

.. code:: javascript
"azure": {
"tenantId": <set from environment>,
"clientId": <set from environment>,
"clientSecret": <set from environment>,
}
``local`` should be substituted with:

.. code:: javascript
"local": { "key": <base64 decoding of LOCAL_MASTERKEY> }
#. If ``autoEncryptOpts`` does not include ``keyVaultNamespace``, default it
to ``keyvault.datakeys``.

#. For each element in ``operations``:

Expand Down Expand Up @@ -194,7 +246,7 @@ Then for each element in ``tests``:
#. For each element in ``outcome``:

- If ``name`` is "collection", create a new MongoClient *without encryption*
and verify that the test collection contains exactly the documents in the
and verify that the test collection contains exactly the documents in the
``data`` array. Ensure this find reads the latest data by using
**primary read preference** with **local read concern** even when the
MongoClient is configured with another read preference or read concern.
Expand Down Expand Up @@ -767,6 +819,6 @@ The following tests that setting ``bypassAutoEncryption=true`` really does bypas
Drivers MAY pass a different value to ``--port`` if they expect their testing infrastructure to be using port 27021. Pass a port that should be free.

#. Use ``client_encrypted`` to insert the document ``{"unencrypted": "test"}`` into ``db.coll``. Expect this to succeed.
#. Use ``client_encrypted`` to insert the document ``{"unencrypted": "test"}`` into ``db.coll``. Expect this to succeed.

#. Validate that mongocryptd was not spawned. Create a MongoClient to localhost:27021 (or whatever was passed via ``--port``) with serverSelectionTimeoutMS=1000. Run an ``isMaster`` command and ensure it fails with a server selection timeout.
225 changes: 225 additions & 0 deletions data/client-side-encryption/awsTemporary.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,225 @@
{
"runOn": [
{
"minServerVersion": "4.1.10"
}
],
"database_name": "default",
"collection_name": "default",
"data": [],
"json_schema": {
"properties": {
"encrypted_w_altname": {
"encrypt": {
"keyId": "/altname",
"bsonType": "string",
"algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Random"
}
},
"encrypted_string": {
"encrypt": {
"keyId": [
{
"$binary": {
"base64": "AAAAAAAAAAAAAAAAAAAAAA==",
"subType": "04"
}
}
],
"bsonType": "string",
"algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic"
}
},
"random": {
"encrypt": {
"keyId": [
{
"$binary": {
"base64": "AAAAAAAAAAAAAAAAAAAAAA==",
"subType": "04"
}
}
],
"bsonType": "string",
"algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Random"
}
},
"encrypted_string_equivalent": {
"encrypt": {
"keyId": [
{
"$binary": {
"base64": "AAAAAAAAAAAAAAAAAAAAAA==",
"subType": "04"
}
}
],
"bsonType": "string",
"algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic"
}
}
},
"bsonType": "object"
},
"key_vault_data": [
{
"status": 1,
"_id": {
"$binary": {
"base64": "AAAAAAAAAAAAAAAAAAAAAA==",
"subType": "04"
}
},
"masterKey": {
"provider": "aws",
"key": "arn:aws:kms:us-east-1:579766882180:key/89fcc2c4-08b0-4bd9-9f25-e30687b580d0",
"region": "us-east-1"
},
"updateDate": {
"$date": {
"$numberLong": "1552949630483"
}
},
"keyMaterial": {
"$binary": {
"base64": "AQICAHhQNmWG2CzOm1dq3kWLM+iDUZhEqnhJwH9wZVpuZ94A8gEqnsxXlR51T5EbEVezUqqKAAAAwjCBvwYJKoZIhvcNAQcGoIGxMIGuAgEAMIGoBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDHa4jo6yp0Z18KgbUgIBEIB74sKxWtV8/YHje5lv5THTl0HIbhSwM6EqRlmBiFFatmEWaeMk4tO4xBX65eq670I5TWPSLMzpp8ncGHMmvHqRajNBnmFtbYxN3E3/WjxmdbOOe+OXpnGJPcGsftc7cB2shRfA4lICPnE26+oVNXT6p0Lo20nY5XC7jyCO",
"subType": "00"
}
},
"creationDate": {
"$date": {
"$numberLong": "1552949630483"
}
},
"keyAltNames": [
"altname",
"another_altname"
]
}
],
"tests": [
{
"description": "Insert a document with auto encryption using the AWS provider with temporary credentials",
"clientOptions": {
"autoEncryptOpts": {
"kmsProviders": {
"awsTemporary": {}
}
}
},
"operations": [
{
"name": "insertOne",
"arguments": {
"document": {
"_id": 1,
"encrypted_string": "string0"
}
}
}
],
"expectations": [
{
"command_started_event": {
"command": {
"listCollections": 1,
"filter": {
"name": "default"
}
},
"command_name": "listCollections"
}
},
{
"command_started_event": {
"command": {
"find": "datakeys",
"filter": {
"$or": [
{
"_id": {
"$in": [
{
"$binary": {
"base64": "AAAAAAAAAAAAAAAAAAAAAA==",
"subType": "04"
}
}
]
}
},
{
"keyAltNames": {
"$in": []
}
}
]
},
"$db": "keyvault"
},
"command_name": "find"
}
},
{
"command_started_event": {
"command": {
"insert": "default",
"documents": [
{
"_id": 1,
"encrypted_string": {
"$binary": {
"base64": "AQAAAAAAAAAAAAAAAAAAAAACwj+3zkv2VM+aTfk60RqhXq6a/77WlLwu/BxXFkL7EppGsju/m8f0x5kBDD3EZTtGALGXlym5jnpZAoSIkswHoA==",
"subType": "06"
}
}
}
],
"ordered": true
},
"command_name": "insert"
}
}
],
"outcome": {
"collection": {
"data": [
{
"_id": 1,
"encrypted_string": {
"$binary": {
"base64": "AQAAAAAAAAAAAAAAAAAAAAACwj+3zkv2VM+aTfk60RqhXq6a/77WlLwu/BxXFkL7EppGsju/m8f0x5kBDD3EZTtGALGXlym5jnpZAoSIkswHoA==",
"subType": "06"
}
}
}
]
}
}
},
{
"description": "Insert with invalid temporary credentials",
"clientOptions": {
"autoEncryptOpts": {
"kmsProviders": {
"awsTemporaryNoSessionToken": {}
}
}
},
"operations": [
{
"name": "insertOne",
"arguments": {
"document": {
"_id": 1,
"encrypted_string": "string0"
}
},
"result": {
"errorContains": "security token"
}
}
]
}
]
}
Loading

0 comments on commit 2c5b75b

Please sign in to comment.