Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New pattern: aws-fargate-opensearch #825

Merged
merged 7 commits into from
Oct 31, 2022
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
lib/*.js
test/*.js
*.d.ts
coverage
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
lib/*.js
test/*.js
*.js.map
*.d.ts
node_modules
*.generated.ts
dist
.jsii

.LAST_BUILD
.nyc_output
coverage
.nycrc
.LAST_PACKAGE
*.snk
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Exclude typescript source and config
*.ts
tsconfig.json
coverage
.nyc_output
*.tgz
*.snk
*.tsbuildinfo

# Include javascript files and typescript declarations
!*.js
!*.d.ts

# Exclude jsii outdir
dist

# Include .jsii
!.jsii

# Include .jsii
!.jsii
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# aws-fargate-opensearch module
<!--BEGIN STABILITY BANNER-->

---

![Stability: Experimental](https://img.shields.io/badge/stability-Experimental-important.svg?style=for-the-badge)

> All classes are under active development and subject to non-backward compatible changes or removal in any
> future version. These are not subject to the [Semantic Versioning](https://semver.org/) model.
> This means that while you may use them, you may need to update your source code when upgrading to a newer version of this package.

---
<!--END STABILITY BANNER-->

| **Reference Documentation**:| <span style="font-weight: normal">https://docs.aws.amazon.com/solutions/latest/constructs/</span>|
|:-------------|:-------------|
<div style="height:8px"></div>

| **Language** | **Package** |
|:-------------|-----------------|
|![Python Logo](https://docs.aws.amazon.com/cdk/api/latest/img/python32.png) Python|`aws_solutions_constructs.aws_fargate_opensearch`|
|![Typescript Logo](https://docs.aws.amazon.com/cdk/api/latest/img/typescript32.png) Typescript|`@aws-solutions-constructs/aws-fargate-opensearch`|
|![Java Logo](https://docs.aws.amazon.com/cdk/api/latest/img/java32.png) Java|`software.amazon.awsconstructs.services.fargateopensearch`|

## Overview
This AWS Solutions Construct implements an AWS Fargate service that can write/read to an Amazon OpenSearch Service domain.

Here is a minimal deployable pattern definition:

Typescript
``` typescript
import { Construct } from 'constructs';
import { Stack, StackProps } from 'aws-cdk-lib';
import { FargateToOpenSearch, FargateToOpenSearchProps } from '@aws-solutions-constructs/aws-fargate-opensearch';

const constructProps: FargateToOpenSearchProps = {
publicApi: true,
ecrRepositoryArn: "arn:aws:ecr:us-east-1:123456789012:repository/your-ecr-repo",
openSearchDomainName: 'testdomain',
// TODO: Ensure the Cognito domain name is globally unique
cognitoDomainName: 'globallyuniquedomain' + Aws.ACCOUNT_ID
};

new FargateToOpenSearch(this, 'test-construct', constructProps);
```

Python
``` python
from aws_solutions_constructs.aws_fargate_opensearch import FargateToOpenSearch, FargateToOpenSearchProps
from aws_cdk import (
Stack
)
from constructs import Construct

FargateToOpenSearch(self, 'test_construct',
public_api=True,
ecr_repository_arn="arn:aws:ecr:us-east-1:123456789012:repository/your-ecr-repo",
open_search_domain_name='testdomain',
# TODO: Ensure the Cognito domain name is globally unique
cognito_domain_name='globallyuniquedomain' + Aws.ACCOUNT_ID)
```

Java
``` java
import software.constructs.Construct;

import software.amazon.awscdk.Stack;
import software.amazon.awscdk.StackProps;
import software.amazon.awsconstructs.services.fargateopensearch.*;

new FargateToOpenSearch(this, "test_construct", new FargateToOpenSearchProps.Builder()
.publicApi(true)
.ecrRepositoryArn("arn:aws:ecr:us-east-1:123456789012:repository/your-ecr-repo"
.openSearchDomainName("testdomain")
// TODO: Ensure the Cognito domain name is globally unique
.cognitoDomainName("globallyuniquedomain" + Aws.ACCOUNT_ID)
.build());
```
## Pattern Construct Props

| **Name** | **Type** | **Description** |
|:-------------|:----------------|-----------------|
| publicApi | `boolean` | Whether the construct is deploying a private or public API. This has implications for the VPC. |
| vpcProps? | [`ec2.VpcProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.VpcProps.html) | Optional custom properties for a VPC the construct will create. This VPC will be used by any Private Hosted Zone the construct creates (that's why loadBalancerProps and privateHostedZoneProps can't include a VPC). Providing both this and existingVpc is an error. |
| existingVpc? | [`ec2.IVpc`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.IVpc.html) | An existing VPC in which to deploy the construct. Providing both this and vpcProps is an error. If the client provides an existing load balancer and/or existing Private Hosted Zone, those constructs must exist in this VPC. |
| clusterProps? | [`ecs.ClusterProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.ClusterProps.html) | Optional properties to create a new ECS cluster. To provide an existing cluster, use the cluster attribute of fargateServiceProps. |
| ecrRepositoryArn? | `string` | The arn of an ECR Repository containing the image to use to generate the containers. Either this or the image property of containerDefinitionProps must be provided. format: arn:aws:ecr:*region*:*account number*:repository/*Repository Name* |
| ecrImageVersion? | `string` | The version of the image to use from the repository. Defaults to 'Latest'. |
| containerDefinitionProps? | [`ecs.ContainerDefinitionProps \| any`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.ContainerDefinitionProps.html) | Optional props to define the container created for the Fargate Service (defaults found in fargate-defaults.ts). |
| fargateTaskDefinitionProps? | [`ecs.FargateTaskDefinitionProps \| any`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.FargateTaskDefinitionProps.html) | Optional props to define the Fargate Task Definition for this construct (defaults found in fargate-defaults.ts). |
| fargateServiceProps? | [`ecs.FargateServiceProps \| any`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.FargateServiceProps.html) | Optional values to override default Fargate Task definition properties (fargate-defaults.ts). The construct will default to launching the service is the most isolated subnets available (precedence: Isolated, Private and Public). Override those and other defaults here. |
| existingFargateServiceObject? | [`ecs.FargateService`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.FargateService.html) | A Fargate Service already instantiated (probably by another Solutions Construct). If this is specified, then no props defining a new service can be provided, including: ecrImageVersion, containerDefinitionProps, fargateTaskDefinitionProps, ecrRepositoryArn, fargateServiceProps, clusterProps.|
| existingContainerDefinitionObject? | [`ecs.ContainerDefinition`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.ContainerDefinition.html) | A container definition already instantiated as part of a Fargate service. This must be the container in the existingFargateServiceObject.|
|openSearchDomainProps?|[`opensearchservice.CfnDomainProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_opensearchservice.CfnDomainProps.html)|Optional user provided props to override the default props for the OpenSearch Service.|
|openSearchDomainName|`string`|Domain name for the OpenSearch Service.|
|cognitoDomainName?|`string`|Optional Amazon Cognito domain name. If omitted the Amazon Cognito domain will default to the OpenSearch Service domain name.|
|createCloudWatchAlarms?|`boolean`|Whether to create the recommended CloudWatch alarms.|
|domainEndpointEnvironmentVariableName?|`string`|Optional name for the OpenSearch Service domain endpoint environment variable set for the Lambda function. Default is `DOMAIN_ENDPOINT`.|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default is not listed on the lambda-opensearch README.md, go ahead and update that on this PR.


## Pattern Properties

| **Name** | **Type** | **Description** |
|:-------------|:----------------|-----------------|
| vpc | [`ec2.IVpc`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.IVpc.html) | The VPC used by the construct (whether created by the construct or provided by the client). |
| service | [`ecs.FargateService`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.FargateService.html) | The AWS Fargate service used by this construct (whether created by this construct or passed to this construct at initialization). |
| container | [`ecs.ContainerDefinition`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ecs.ContainerDefinition.html) | The container associated with the AWS Fargate service in the service property. |
|userPool|[`cognito.UserPool`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.UserPool.html)|Returns an instance of `cognito.UserPool` created by the construct.|
|userPoolClient|[`cognito.UserPoolClient`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.UserPoolClient.html)|Returns an instance of `cognito.UserPoolClient` created by the construct.|
|identityPool|[`cognito.CfnIdentityPool`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.CfnIdentityPool.html)|Returns an instance of `cognito.CfnIdentityPool` created by the construct.|
|openSearchDomain|[`opensearchservice.CfnDomain`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_opensearchservice.CfnDomain.html)|Returns an instance of `opensearch.CfnDomain` created by the construct.|
|openSearchRole|[`iam.Role`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_iam.Role.html)|Returns an instance of `iam.Role` created by the construct for `opensearch.CfnDomain`.|
|cloudWatchAlarms?|[`cloudwatch.Alarm[]`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cloudwatch.Alarm.html)|Returns a list of `cloudwatch.Alarm` created by the construct.|

## Default settings

Out of the box implementation of the Construct without any override will set the following defaults:

### AWS Fargate Service
* Sets up an AWS Fargate service
* Uses the existing service if provided
* Creates a new service if none provided
* Service will run in isolated subnets if available, then private subnets if available and finally public subnets
* Adds environment variables to the container with the OpenSearch Service domain endpoint
* Add permissions to the container IAM role allowing it to write/read to the OpenSearch Service domain endpoint

### Amazon Cognito
* Set password policy for User Pools
* Enforce the advanced security mode for User Pools

### Amazon OpenSearch Service
* Deploy best practices CloudWatch Alarms for the OpenSearch Service domain
* Secure the OpenSearch Service dashboard access with Cognito User Pools
* Enable server-side encryption for OpenSearch Service domain using AWS managed KMS Key
* Enable node-to-node encryption for the OpenSearch Service domain
* Configure the cluster for the OpenSearch Service domain

## Architecture
![Architecture Diagram](architecture.png)

***
&copy; Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
/**
* Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
* with the License. A copy of the License is located at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* or in the 'license' file accompanying this file. This file is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES
* OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions
* and limitations under the License.
*/

import * as defaults from "@aws-solutions-constructs/core";
import * as ecs from "aws-cdk-lib/aws-ecs";
import * as iam from "aws-cdk-lib/aws-iam";
import * as cognito from 'aws-cdk-lib/aws-cognito';
import * as ec2 from "aws-cdk-lib/aws-ec2";
import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';
import * as opensearch from "aws-cdk-lib/aws-opensearchservice";
// Note: To ensure CDKv2 compatibility, keep the import statement for Construct separate
import { Construct } from "constructs";

export interface FargateToOpenSearchProps {
/**
* Optional custom properties for a VPC the construct will create. This VPC will
* be used by the new Fargate service the construct creates (that's
* why targetGroupProps can't include a VPC). Providing
* both this and existingVpc is an error.
*
* @default - none
*/
readonly vpcProps?: ec2.VpcProps;
/**
* An existing VPC in which to deploy the construct. Providing both this and
* vpcProps is an error. If the client provides an existing Fargate service,
* this value must be the VPC where the service is running.
*
* @default - none
*/
readonly existingVpc?: ec2.IVpc;
/**
* Whether the construct is deploying a private or public API. This has implications for the VPC deployed
* by this construct.
*
* @default - none
*/
readonly publicApi: boolean;
/**
* Optional properties to create a new ECS cluster
*/
readonly clusterProps?: ecs.ClusterProps;
/**
* The arn of an ECR Repository containing the image to use
* to generate the containers
*
* format:
* arn:aws:ecr:[region]:[account number]:repository/[Repository Name]
*/
readonly ecrRepositoryArn?: string;
/**
* The version of the image to use from the repository
*
* @default - 'latest'
*/
readonly ecrImageVersion?: string;
/*
* Optional props to define the container created for the Fargate Service
*
* defaults - fargate-defaults.ts
*/
readonly containerDefinitionProps?: ecs.ContainerDefinitionProps | any;
/*
* Optional props to define the Fargate Task Definition for this construct
*
* defaults - fargate-defaults.ts
*/
readonly fargateTaskDefinitionProps?: ecs.FargateTaskDefinitionProps | any;
/**
* Optional values to override default Fargate Task definition properties
* (fargate-defaults.ts). The construct will default to launching the service
* is the most isolated subnets available (precedence: Isolated, Private and
* Public). Override those and other defaults here.
*
* defaults - fargate-defaults.ts
*/
readonly fargateServiceProps?: ecs.FargateServiceProps | any;
/**
* A Fargate Service already instantiated (probably by another Solutions Construct). If
* this is specified, then no props defining a new service can be provided, including:
* existingImageObject, ecrImageVersion, containerDefintionProps, fargateTaskDefinitionProps,
* ecrRepositoryArn, fargateServiceProps, clusterProps, existingClusterInterface. If this value
* is provided, then existingContainerDefinitionObject must be provided as well.
*
* @default - none
*/
readonly existingFargateServiceObject?: ecs.FargateService;
/*
* A container definition already instantiated as part of a Fargate service. This must
* be the container in the existingFargateServiceObject.
*
* @default - None
*/
readonly existingContainerDefinitionObject?: ecs.ContainerDefinition;
/**
* Optional user provided props to override the default props for the OpenSearch Service.
*
* @default - Default props are used
*/
readonly openSearchDomainProps?: opensearch.CfnDomainProps;
/**
* Domain name for the OpenSearch Service.
*
* @default - None
*/
readonly openSearchDomainName: string;
/**
* Optional Amazon Cognito domain name. If omitted the Amazon Cognito domain will default to the OpenSearch Service domain name.
*
* @default - the OpenSearch Service domain name
*/
readonly cognitoDomainName?: string;
/**
* Whether to create recommended CloudWatch alarms
*
* @default - Alarms are created
*/
readonly createCloudWatchAlarms?: boolean;
/**
* Optional Name for the container environment variable set to the domain endpoint.
*
* @default - DOMAIN_ENDPOINT
*/
readonly domainEndpointEnvironmentVariableName?: string;
}

export class FargateToOpenSearch extends Construct {
public readonly vpc: ec2.IVpc;
public readonly service: ecs.FargateService;
public readonly container: ecs.ContainerDefinition;
public readonly userPool: cognito.UserPool;
public readonly userPoolClient: cognito.UserPoolClient;
public readonly identityPool: cognito.CfnIdentityPool;
public readonly openSearchDomain: opensearch.CfnDomain;
public readonly openSearchRole: iam.Role;
public readonly cloudWatchAlarms?: cloudwatch.Alarm[];

constructor(scope: Construct, id: string, props: FargateToOpenSearchProps) {
super(scope, id);
defaults.CheckProps(props);
defaults.CheckFargateProps(props);

this.vpc = defaults.buildVpc(scope, {
existingVpc: props.existingVpc,
defaultVpcProps: props.publicApi ? defaults.DefaultPublicPrivateVpcProps() : defaults.DefaultIsolatedVpcProps(),
userVpcProps: props.vpcProps,
constructVpcProps: { enableDnsHostnames: true, enableDnsSupport: true }
});

if (props.existingFargateServiceObject) {
this.service = props.existingFargateServiceObject;
// CheckFargateProps confirms that the container is provided
this.container = props.existingContainerDefinitionObject!;
} else {
[this.service, this.container] = defaults.CreateFargateService(
scope,
id,
this.vpc,
props.clusterProps,
props.ecrRepositoryArn,
props.ecrImageVersion,
props.fargateTaskDefinitionProps,
props.containerDefinitionProps,
props.fargateServiceProps
);
}

let cognitoAuthorizedRole: iam.Role;

[this.userPool, this.userPoolClient, this.identityPool, cognitoAuthorizedRole] =
defaults.buildCognitoForSearchService(this, props.cognitoDomainName ?? props.openSearchDomainName);

let securityGroupIds;

if (this.vpc) {
securityGroupIds = defaults.getServiceVpcSecurityGroupIds(this.service);
}

const buildOpenSearchProps: defaults.BuildOpenSearchProps = {
userpool: this.userPool,
identitypool: this.identityPool,
cognitoAuthorizedRoleARN: cognitoAuthorizedRole.roleArn,
vpc: this.vpc,
openSearchDomainName: props.openSearchDomainName,
clientDomainProps: props.openSearchDomainProps,
securityGroupIds
};

[this.openSearchDomain, this.openSearchRole] = defaults.buildOpenSearch(this, buildOpenSearchProps);

if (props.createCloudWatchAlarms === undefined || props.createCloudWatchAlarms) {
this.cloudWatchAlarms = defaults.buildOpenSearchCWAlarms(this);
}

// Add environment variables
const bucketArnEnvironmentVariableName = props.domainEndpointEnvironmentVariableName || 'DOMAIN_ENDPOINT';
mickychetta marked this conversation as resolved.
Show resolved Hide resolved
this.container.addEnvironment(bucketArnEnvironmentVariableName, this.openSearchDomain.attrDomainEndpoint);

}
}
Loading