-
Notifications
You must be signed in to change notification settings - Fork 11
4. Resources Meta Descriptions
Navigation:
4.2.2 DynamoDB Global Secondary Index
4.5.3 API Gateway Resource (path)
4.5.4 API Gateway Resource Method
4.5.5 Ways to deploy API Gateway to AWS Account with Syndicate
4.13 Step Functions State Machine
4.14.2 Amazon Cognito Identity Pools
4.18 Batch Compute Environment Instance
This guide contains command samples to generate meta, JSON Meta generated by the CLI command and the examples for each type of the supported resources.
Keep in mind that some resources are co-dependent, so pay attention to PRECONDITIONS/LINKED RESOURCES sections below descriptions.
Useful links contain detailed instructions on resource management, so feel free to check them out.
syndicate generate lambda --name "string" --runtime "java"|"nodejs"|"python"|"dotnet" --project_path "string"-
--name(string) [REQUIRED, MULTIPLE] - The lambda function name. -
--runtime(string) [REQUIRED] - Lambda's runtime. If multiple lambda names are specified, the runtime will be applied to all lambdas. -
--project_path(string) - Path to the project folder. Default value: the one from the current config if it exists. Otherwise - the current working directory.
{
"version": "string",
"name": "string",
"lambda_path": "string",
"func_name": "string",
"resource_type": "string",
"iam_arn_role": "string",
"runtime": "nodejs"|"nodejs4.3"|"nodejs6.10"|"nodejs8.10"|"nodejs10.x"|"nodejs12.x"|"nodejs14.x"|"nodejs16.x"|"java8"|"java8.al2"|"java11"|"python2.7"|"python3.6"|"python3.7"|"python3.8"|"python3.9"|"dotnet8"|"nodejs4.3-edge"|"nodejs18.x"|"python3.10"|"java17"|"python3.11"|"nodejs20.x"|"python3.12"|"java21",
"memory": 123,
"timeout": 123,
"architectures": ["x86_64"|"arm64"],
"logs_expirations": 1|3|5|7|14|30|60|90|120|150|180|365|400|545|731|1827|3653,
"deployment_package": "string",
"max_concurrency": 123,
"provisioned_concurrency": {
"qualifier": "ALIAS"|"VERSION",
"value": 123
},
"dependencies": [
{
"resource_type": "string",
"resource_name": "string"
}
],
"event_sources": [
{
"resource_type": "s3_trigger",
"target_bucket": "string",
"s3_events": [
"s3:ReducedRedundancyLostObject"|"s3:ObjectCreated:*"|"s3:ObjectCreated:Put"|"s3:ObjectCreated:Post"|"s3:ObjectCreated:Copy"|"s3:ObjectCreated:CompleteMultipartUpload"|"s3:ObjectRemoved:*"|"s3:ObjectRemoved:Delete"|"s3:ObjectRemoved:DeleteMarkerCreated"
],
"filter_rules": [
{
"Name": "prefix"|"suffix",
"Value": "string"
}
]
},
{
"resource_type": "sqs_trigger",
"target_queue": "string",
"batch_size": 123,
"batch_window": 123,
"function_response_types": ["ReportBatchItemFailures"]
},
{
"resource_type": "dynamodb_trigger",
"target_table": "string",
"batch_size": 123,
"function_response_types": ["ReportBatchItemFailures"]
},
{
"resource_type": "cloudwatch_rule_trigger",
"target_rule": "string"
},
{
"resource_type": "sns_topic_trigger",
"target_topic": "string",
"region": "all"|["us-east-1"|"us-east-2"|"us-west-1"|"us-west-2"|"eu-west-1"|"eu-west-2"|"eu-west-3"|"eu-central-1"|"ca-central-1"|"ap-south-1"|"ap-southeast-1"|"ap-southeast-2"|"ap-northeast-1"|"ap-northeast-2"|"eu-north-1"]
},
{
"resource_type": "kinesis_trigger",
"target_stream": "string",
"batch_size": 123,
"starting_position": "LATEST"|"TRIM_HORIZON"|"AT_TIMESTAMP",
"function_response_types": ["ReportBatchItemFailures"]
}
],
"env_variables": {
"string": "string"|{
"resource_name": "string",
"resource_type": "string",
"parameter": "string"
}
},
"subnet_ids": ["string"],
"security_group_ids": ["string"],
"tracing_mode": "Active"|"PassThrough",
"publish_version": true|false,
"alias": "string",
"dl_resource_type": "sns"|"sqs",
"dl_resource_name": "string",
"layers": ["string"],
"url_config": {
"auth_type": "NONE"|"AWS_IAM",
"cors": {
"allow_credentials": true|false,
"allow_headers": ["string"],
"allow_methods": ["string"],
"allow_origins": ["string"],
"expose_headers": ["string"],
"max_age": 123
},
"principal": "string",
"source_arn": "string"
},
"ephemeral_storage": 123,
"max_retries": 123
}
-
version (string) - Lambda version. Is used at artifact build. Required for Python runtime.
-
name (string) - Name of the Lambda. Required for Python runtime.
-
lambda_path (string) - The relative path to the Lambda handler within the Python project. Required for Python runtime.
-
func_name (string) [REQUIRED] - Function handler name.
-
resource_type (string) [REQUIRED] - Resource type.
-
iam_arn_role (string) [REQUIRED] - Lambda execution role name.
-
runtime (string) [REQUIRED] - Lambda executive environment.
-
memory (int) [REQUIRED] - Lambda memory.
-
timeout (int) [REQUIRED] - The function execution time at which Lambda should terminate the function.
-
architectures (list) - The function processor architecture.
-
logs_expirations (int) - Lambda logs retention in days. In case of this parameter not specified CW log group will not be automatically created.
-
deployment_package (string) - Name of the artifact. The parameter is required for Java. Example:
component.jar -
max_concurrency (int) - Maximum number of simultaneous executions for Lambda function.
-
provisioned_concurrency (dict) - Concurrency configuration applied to Lambda function.
-
qualifier (string) - Accepted values:
-
ALIAS - provisioned concurrency configuration will be set to the function alias. Target alias for the configuration will be obtained from field 'alias' of lambda meta.
-
VERSION - provisioned concurrency configuration will be applied to the function version. Target version is published one from the
$LATEST.
-
-
value (string) - Value of provisioned concurrency executions that will be applied to the function.
-
-
dependencies (list) - List of resources that Lambda is dependent on.
-
event_sources (list) - List of dicts that describe Lambda triggers.
-
resource_type (string) - Resource type of the trigger. Triggers by types:
-
DynamoDB trigger
-
resource_type (string) [REQUIRED] - Resource type -
dynamodb_trigger. -
target_table (string) [REQUIRED] - Name of the table to be triggered by.
-
batch_size (int) [REQUIRED] - Count of entities to process during one Lambda call.
-
function_response_types (list) - A list of current response type enums applied to the event source mapping.
-
-
CloudWatch Rule trigger
-
resource_type (string) [REQUIRED] - Resource type -
cloudwatch_rule_trigger. -
target_rule (string) [REQUIRED] - CloudWatch rule name.
-
-
S3 event trigger
-
resource_type (string) [REQUIRED] - Resource type -
s3_trigger. -
target_bucket (string) [REQUIRED] - Name of the S3 bucket.
-
s3_events (list) [REQUIRED] - List of the s3 events.
-
filter_rules (list) - List of the object key filters by prefix or suffix.
-
Name (string) - Type of the filter.
-
Value (string) - Value of the filter
-
-
-
SNS topic trigger
-
resource_type (string) [REQUIRED] - Resource type -
sns_topic_trigger. -
target_topic (string) [REQUIRED] - The name of the SNS topic to which the Lambda is subscribed.
-
region (string) - The name of the region in which the topic is deployed.
-
-
Kinesis stream trigger
-
resource_type (string) [REQUIRED] - Resource type -
kinesis_trigger. -
target_stream (string) [REQUIRED] - Name of the Kinesis stream.
-
batch_size (int) [REQUIRED] - The quantity of the entries processed in one Lambda call.
-
starting_position (string) [REQUIRED] - The position at which the entry processing starts.
-
function_response_types (list) - A list of current response type enums applied to the event source mapping.
-
-
SQS trigger
-
resource_type (string) [REQUIRED] - Resource type -
sqs_trigger. -
target_queue (string) [REQUIRED] - Name of the SQS queue.
-
batch_size (int) [REQUIRED] - The quantity of entries, processed during one Lambda call.
-
batch_window (int) - The maximum amount of time, in seconds, that Lambda spends gathering records before invoking the function. For streams and Amazon SQS event sources, when you set batch_size to a value greater than 10, batch_window must be set to at least 1.
-
function_response_types (list) - A list of current response type enums applied to the event source mapping.
-
-
-
-
env_variables (dict) - Map of the environment variables. Value can be either of string or dict type:
-
<variable_name> (str) Static environment variable format
-
<variable_name> (dict) - Dynamic environment variable format to extract value from existing resource parameter
Note: Don't forget to include the resource you are getting the value from in the lambda dependencies.
-
resource_name (str) [REQUIRED] - Resource name
-
resource_type (str) [REQUIRED] - Resource type
-
parameter (str) [REQUIRED] - Resource parameter name
Examples:
Cognito user pool ID as lambda environment variable value:
"demo_cup_id": { "resource_name": "demo_cup", "resource_type": "cognito_idp", "parameter": "id" }Cognito user pool client ID as lambda environment variable value:
"demo_cup_id": { "resource_name": "demo_cup", "resource_type": "cognito_idp", "parameter": "client_id" } -
-
-
subnet_ids (list) - A list of VPC subnet IDs. Required parameter to connect lambda function to a VPC.
-
security_group_ids (list) - A list of VPC security groups IDs. Required parameter to connect lambda function to a VPC.
-
tracing_mode (string) - Set Mode to Active to sample and trace a subset of incoming requests with X-Ray.
-
publish_version (boolean) - Publishes the first version of the function during creation if the parameter is set to true.
-
alias (string) - Alias name that is a pointer to a function version.
-
dl_resource_type (string) - Set Amazon SQS queue or Amazon SNS topic for a dead-letter queue. Must be specified with the
dl_resource_nameparameter. -
dl_resource_name (string) - SQS queue name or SNS topic name. Must be specified with the
dl_resource_typeparameter. -
layers (list) - A list function layer's names to add to the function's execution environment.
-
url_config (dict) - Lambda url configuration
-
auth_type (string) - The type of authentication that your function URL uses. Set to
AWS_IAMif you want to restrict access to authenticated users only. Set toNONEif you want to bypass IAM authentication to create a public endpoint with a function url. -
principal (string) - Required for
AWS_IAMauth type. Principal which will be applied to lambda's resource-based-policy. -
source_arn (string) - Required for
AWS_IAMauth type. Source ARN which will be applied to lambda's resource-based-policy. -
cors (list) - Contain information about CORS.
-
allow_credentials (boolean) - Whether to allow cookies or other credentials in requests to your function URL. The default is
false. -
allow_headers (list) - The HTTP headers that origins can include in requests to your function URL. For example:
Date,Keep-Alive,X-Custom-Header. -
allow_methods (list) - The HTTP methods that are allowed when calling your function URL. For example:
GET,POST,DELETE, or the wildcard character (*). -
allow_origins (list) - The origins that can access your function URL. You can list any number of specific origins, separated by a comma. For example:
https://www.example.com,http://localhost:60905. Alternatively, you can grant access to all origins using the wildcard character (*). -
expose_headers (list) - The HTTP headers in your function response that you want to expose to origins that call your function URL. For example:
Date,Keep-Alive,X-Custom-Header. -
max_age (int) - The maximum amount of time, in seconds, that web browsers can cache results of a preflight request. By default, this is set to
0, which means that the browser doesn’t cache results.
-
-
-
ephemeral_storage (int) - The size of the function’s
/tmpdirectory in MB. The default value is512, but can be any whole number between512and10240MB. -
max_retries (int) - The maximum number of times to retry when the asynchronously configured function returns an error. Default value is
2.
syndicate generate lambda_layer --name "string" --runtime "java"|"nodejs"|"python"|"dotnet" --link_with_lambda "string" --project_path "string"-
--name(string) [REQUIRED] - Lambda layer name. -
--runtime(string) [REQUIRED] - Lambda layer runtime. -
--link_with_lambda(string) [REQUIRED, MULTIPLE] - Lambda function name to link the layer with. -
--project_path(string) - Path to the project folder. Default value: the one from the current config if it exists. Otherwise - the current working directory.
{
"name": "string",
"resource_type": "lambda_layer",
"runtimes": ["nodejs"|"nodejs4.3"|"nodejs6.10"|"nodejs8.10"|"nodejs10.x"|"nodejs12.x"|"nodejs14.x"|"nodejs16.x"|"java8"|"java8.al2"|"java11"|"python2.7"|"python3.6"|"python3.7"|"python3.8"|"python3.9"|"dotnet8"|"nodejs4.3-edge"|"nodejs18.x"|"python3.10"|"java17"|"python3.11"|"nodejs20.x"|"python3.12"|"java21"],
"architectures": ["x86_64"|"arm64"],
"deployment_package": "string",
"custom_packages": ["string"]
}
-
name (string) [REQUIRED] - Name of the layer.
-
resource_type (string) [REQUIRED] - Resource type -
lambda_layer. -
runtimes (list) [REQUIRED] - A list of compatible function runtimes.
-
architectures (list) - A list of compatible instruction set architectures.
-
deployment_package (string) [REQUIRED] - Deployment package name of the layer. For example:
lambda_layer.zip. -
custom_packages (string) [Applicable only for runtime dotnet] - A list of file names with the layer custom NuGet packages. Files with custom packages(.nupkg) have to be placed in the layer root directory.
Resource type: Dynamo DB table
Command to generate meta:
syndicate generate meta dynamodb
--resource_name DemoTable
--hash_key_name Id
--hash_key_type N
--sort_key_name Category
--sort_key_type S
--read_capacity 1
--write_capacity 1
JSON Meta generated by the CLI command (deployment_resources.json):
{
"DemoTable": {
"resource_type": "dynamodb_table",
"hash_key_name": "Id",
"hash_key_type": "N",
"sort_key_name": "Category",
"sort_key_type": "S",
"read_capacity": 1,
"write_capacity": 1,
"global_indexes": [],
"autoscaling": []
}
}
Parameters:
- resource_type (string) [REQUIRED] Table name.
- hash_key_name (string) [REQUIRED] Table hash key.
- hash_key_type (string) [REQUIRED] Hash key type.
- sort_key_name (string) Developer provider name
- sort_key_type (string) Table sort key. If not specified, the table will have only a hash key.
- read_capacity (int) The maximum number of strongly consistent reads that can be performed per second. If not specified, sets the default value to 1.
- write_capacity (int) The maximum number of writing processes consumed per second. If not specified, sets the default value to 1.
- global_indexes (list) Table indexes. May contain several objects.
- “autoscaling” (list) Table autoscaling configuration.
-
“external” (bool)
Mark of an external AWS resource. Syndicate doesn't try to create external resources.
Instead, it checks if such resources exist and their metadata match.
To force syndicate clean external resources, use
--clean_externalflag. Syndicate will also treat the existing dynamodb table as external without specifying this flag. - “ttl_attribute_name” (string) The name of the TTL attribute used to store the expiration time for items in the table.
"table_name": {
"resource_type": "string",
"hash_key_name": "string",
"hash_key_type": "S/N/B",
"sort_key_name": "string",
"sort_key_type": "S/N/B",
"read_capacity": int,
"write_capacity": int,
"global_indexes": [
{}, {}
],
“autoscaling”: [
{}, {}
],
"external": true,
"ttl_attribute_name": "string"
}
NOTE: Here we have a Dynamo DB table description. "sort_key_name" and "sort_key_type" are not required because a table can be created without a sort key definition.
Example:
"example_table": {
"resource_type": "dynamodb_table",
"hash_key_name": "n",
"hash_key_type": "S",
"read_capacity": 5,
"write_capacity": 1,
"global_indexes": [
{
"name": "example_index",
"index_key_name": "ex_idx",
"index_key_type": "S"
}
],
"autoscaling": [
{
"resource_name": "example_table",
"min_capacity": 1,
"max_capacity": 100,
"role_name": "example_dynamodb_table_role",
"config": {
"target_utilization": 75,
"policy_name": "rcu_ex_table_policy"
},
"dimension": "dynamodb:table:ReadCapacityUnits"
},
{
"resource_name": "example_table",
"min_capacity": 1,
"max_capacity": 100,
"role_name": "example_dynamodb_table_role",
"config": {
"policy_name": "wcu_ex_table_policy",
"target_utilization": 50.0,
"scale_out_cooldown": 10,
"scale_in_cooldown": 60
},
"dimension": "dynamodb:table:WriteCapacityUnits"
},
{
"resource_name": "example_index",
"min_capacity": 1,
"max_capacity": 100,
"role_name": "example_dynamodb_table_role",
"config": {
"target_utilization": 75,
"policy_name": "rcu_example_index_policy"
},
"dimension": "dynamodb:index:ReadCapacityUnits"
},
{
"resource_name": "example_index",
"min_capacity": 1,
"max_capacity": 100,
"role_name": "example_dynamodb_table_role",
"config": {
"policy_name": "wcu_example_index_policy",
"target_utilization": 50.0
},
"dimension": "dynamodb:index:WriteCapacityUnits"
}
]
}
Resource type: DynamoDB Global Secondary Index (for previously generated DynamoDB Table)
Command to generate meta:
syndicate generate meta dynamodb_global_index
--table_name DemoTable
--name CategoryIndex
--index_key_name Category
--index_key_type S
--index_sort_key_name Score
--index_sort_key_type N
JSON Meta modified by the CLI command (deployment_resources.json):
{
"DemoTable": {
"resource_type": "dynamodb_table",
"hash_key_name": "Id",
"hash_key_type": "N",
"sort_key_name": "Category",
"sort_key_type": "S",
"read_capacity": 1,
"write_capacity": 1,
"global_indexes": [
{
"name": "CategoryIndex",
"index_key_name": "Category",
"index_key_type": "S",
"index_sort_key_name": "Score",
"index_sort_key_type": "N"
}
],
"autoscaling": []
}
}
Linked resources: dynamodb_table: 4.2.1 Dynamo DB table
Resource type: DynamoDB Autoscaling (for previously generated DynamoDB Table)
Command to generate meta:
syndicate generate meta dynamodb_autoscaling
--table_name DemoTable
--policy_name DemoScalingPolicy
--min_capacity 1
--max_capacity 10
--target_utilization 70
--scale_in_cooldown 60
--scale_out_cooldown 60
--dimension dynamodb:table:ReadCapacityUnits
--role_name AWSServiceRoleForApplicationAutoScaling_DynamoDBTable
JSON Meta modified by the CLI command (deployment_resources.json):
{
"DemoTable": {
"resource_type": "dynamodb_table",
"hash_key_name": "Id",
"hash_key_type": "N",
"sort_key_name": "Category",
"sort_key_type": "S",
"read_capacity": 1,
"write_capacity": 1,
"global_indexes": [
{
"name": "CategoryIndex",
"index_key_name": "Category",
"index_key_type": "S",
"index_sort_key_name": "Score",
"index_sort_key_type": "N"
}
],
"autoscaling": [
{
"resource_name": "DemoTable",
"role_name": "AWSServiceRoleForApplicationAutoScaling_DynamoDBTable",
"min_capacity": 1,
"max_capacity": 10,
"config": {
"policy_name": "DemoScalingPolicy",
"target_utilization": 70,
"scale_in_cooldown": 60,
"scale_out_cooldown": 60
},
"dimension": "dynamodb:table:ReadCapacityUnits"
}
]
}
}
Linked resources: dynamodb_table: 4.2.1 DynamoDB table iam_role: 4.10 IAM Role
Resource type: CloudWatch Event Rule
Command to generate meta:
syndicate generate meta cloudwatch_event_rule
--resource_name demo-cloudwatch-event-rule
--rule_type schedule
--expression "rate(1 minute)"
--region eu-central-1
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-cloudwatch-event-rule": {
"resource_type": "cloudwatch_rule",
"rule_type": "schedule",
"region": "eu-central-1",
"expression": "rate(1 minute)"
}
}
- schedule
"rule_name": {
"resource_type": "string",
"rule_type": "string",
"expression": "string",
"region": /"all"/"region_name"/["region_name1", ..]
}
- resource_type (string) [REQUIRED] Resource type.
- rule_type (string) [REQUIRED] Rule type.
- expression (string) [REQUIRED] Rule expression (cron schedule).
- region (string/list) The region where the rule is deployed. If not specified ["region_name1", ..] the default value is taken from sdct.conf.
- ec2
"rule_name": {
"resource_type": "string",
"rule_type": "string",
"instance_ids": [list],
"instance_states": [list],
"region": /"all"/"region_name"/["region_name1", ..]
}
- resource_type (string) [REQUIRED] Resource type.
- rule_type (string) [REQUIRED] Rule type.
- instance_ids (list) The list of EC2 instances, to which the rule is bound. If not specified, the default value is set to 'any'.
- instance_states (list) EC2 instance states, which are monitored by the rule. If not specified, the default value is set to 'any'.
- region (string/list) The region where the rule is deployed. If not specified ["region_name1", ..] the default value is taken from sdct.conf.
- api call
"rule_name": {
"resource_type": "string",
"rule_type": "string",
"operations": [list],
"region": /"all"/"region_name"/["region_name1", ..]
}
- resource_type (string) [REQUIRED] Resource type.
- rule_type (string) [REQUIRED] Rule type.
- aws_service (string) [REQUIRED] The name of the AWS service, which the rule listens to.
- operations (list) The actions monitored by rule. If not specified, the default value is set to 'any'.
- region (string/list) The region where the rule is deployed. If not specified ["region_name1", ..] the default value is taken from sdct.conf.
Example:
"weekly_report_event": {
"rule_type": "schedule",
"expression": "cron(0 8? * MON *)",
"resource_type": "cloudwatch_rule"
}
Resource type: S3 Bucket
Command to generate meta:
syndicate generate meta s3_bucket
--resource_name demo-bucket
--location eu-central-1
--acl private
--block_public_acls true
--ignore_public_acls true
--block_public_policy true
--restrict_public_buckets true
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-bucket": {
"resource_type": "s3_bucket",
"acl": "private",
"location": "eu-central-1",
"cors": [],
"policy": {},
"public_access_block": {
"block_public_acls": true,
"ignore_public_acls": true,
"block_public_policy": true,
"restrict_public_buckets": true
}
}
}
Parameters:
- resource_type (string) [REQUIRED] Resource type.
- location (string) The region, where the bucket is created, the default value is the region set in sdct.conf*
- acl (string) The canned ACL to be applied to the bucket.
- policy (string) IAM policy to be attached to the bucket.
- LifecycleConfiguration (string) S3 Lifecycle configuration
- static_website_hosting (boolean) Preconfiguring the S3 bucket for static website hosting
"bucket_name": {
"resource_type": "s3_bucket",
"location": "eu-west-1|us-west-1|us-west-2|ap-south-1|ap-southeast-1|ap-southeast-2|ap-northeast-1|sa-east-1|cn-north-1|eu-central-1"
"acl": "private|public-read|public-read-write|authenticated-read",
"policy": {
"Version": "2012-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"Statement": [
{
"Action": "s3:GetObject",
"Principal": "*",
"Resource": "arn:aws:s3:::${ui_bucket}",
"Effect": "Allow",
"Sid": "1"
}
]
},
"LifecycleConfiguration": {
"Rules": [
{
"Expiration": {
"Date": "datetime(2015,1,1)",
"Days": 123,
"ExpiredObjectDeleteMarker": "True|False"
},
"ID": "string",
"Prefix": "string",
"Filter": {
"Prefix": "string",
"Tag": {
"Key": "string",
"Value": "string"
},
"ObjectSizeGreaterThan": 123,
"ObjectSizeLessThan": 123,
"And": {
"Prefix": "string",
"Tags": [
{
"Key": "string",
"Value": "string"
}
],
"ObjectSizeGreaterThan": 123,
"ObjectSizeLessThan": 123
}
},
"Status": "Enabled|Disabled",
"Transitions": [
{
"Date": "datetime(2015,1,1)",
"Days": 123,
"StorageClass": "GLACIER|STANDARD_IA|ONEZONE_IA|INTELLIGENT_TIERING|DEEP_ARCHIVE|GLACIER_IR"
}
],
"NoncurrentVersionTransitions": [
{
"NoncurrentDays": 123,
"StorageClass": "GLACIER|STANDARD_IA|ONEZONE_IA|INTELLIGENT_TIERING|DEEP_ARCHIVE|GLACIER_IR",
"NewerNoncurrentVersions": 123
}
],
"NoncurrentVersionExpiration": {
"NoncurrentDays": 123,
"NewerNoncurrentVersions": 123
},
"AbortIncompleteMultipartUpload": {
"DaysAfterInitiation": 123
}
}
]
}
}
Example:
"${ui_bucket}": {
"policy": {
"Version": "2012-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"Statement": [
{
"Action": "s3:GetObject",
"Principal": "*",
"Resource": "arn:aws:s3:::{bucket_name}/*",
"Effect": "Allow",
"Sid": "1"
}
]
},
"resource_type": "s3_bucket",
"acl": "public-read",
"LifecycleConfiguration": {
"Rules": [
{
"ID": "TestId",
"Status": "Enabled",
"Expiration": {
"Days": 1825
},
"Filter": {
"Prefix": "documents/"
},
"Transitions": [
{
"Days": 365,
"StorageClass": "GLACIER"
}
]
}
]
}
}
Resource type: API Gateway
Command to generate meta:
syndicate generate meta api_gateway
--resource_name demo-api
--deploy_stage api
--minimum_compression_size 0
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-api": {
"resource_type": "api_gateway",
"deploy_stage": "api",
"dependencies": [],
"resources": {},
"minimum_compression_size": 0
}
}
Parameters:
- resource_type (string) [REQUIRED] Resource type.
- deploy_stage (string) [REQUIRED] The stage of the deployed API.
- resources (map) [REQUIRED]
- cluster_throttling_configuration (map) The stage throttling configuration
- cluster_cache_configuration (map) The stage cache configuration
Linked resources: api_gateway_resource: 4.5.2 API Gateway Resource
Resource type: API Gateway Custom Authorizer
Command to generate meta:
syndicate generate meta api_gateway_authorizer
--api_name demo-api
--name demo-authorizer
--type COGNITO_USER_POOLS
--provider_name demo-cognito-user-pool
JSON Meta generated by the CLI command (deployment_resources.json):
"demo-api": {
"resource_type": "api_gateway",
"deploy_stage": "api",
"dependencies": [],
"resources": {},
"minimum_compression_size": 0,
"authorizers": {
"demo-authorizer": {
"type": "COGNITO_USER_POOLS",
"identity_source": "method.request.header.Authorization",
"ttl": 300,
"user_pools": [
"demo-cognito-user-pool"
]
}
}
}
Precondition: API Gateway and Cognito User Pool resources should be previously specified.
Linked resources: api_gateway: 4.5.1 API Gateway, cognito_idp: 4.14.1 Cognito User Pools
Resource type: API Gateway Resource (path)
Command to generate meta:
syndicate generate meta api_gateway_resource
--api_name demo-api
--path /demo
--enable_cors false
JSON Meta modified by the CLI command (deployment_resources.json):
{
"demo-api": {
"resource_type": "api_gateway",
"deploy_stage": "api",
"dependencies": [],
"resources": {
"/demo": {
"enable_cors": false
}
},
"minimum_compression_size": 0
}
}
Parameters:
- enable_cors (boolean) Enables CORS on the resource methods.
Resource type: API Gateway Resource Method
Command to generate meta:
syndicate generate meta api_gateway_resource_method
--api_name demo-api
--path /demo
--method GET
--integration_type mock
--authorization_type AWS_IAM
--api_key_required false
JSON Meta modified by the CLI command (deployment_resources.json):
{
"demo-api": {
"resource_type": "api_gateway",
"deploy_stage": "api",
"dependencies": [],
"resources": {
"/demo": {
"enable_cors": false,
"GET": {
"authorization_type": "AWS_IAM",
"integration_type": "mock",
"api_key_required": false,
"method_request_parameters": {},
"integration_request_body_template": {},
"responses": [],
"integration_responses": [],
"default_error_pattern": true
}
}
},
"minimum_compression_size": 0
}
}
Parameters:
- authorization_type (string) The method's authorization type (sets the default value to 'NONE').
- api_key_required (boolean) Specifies whether the method required a valid ApiKey (the default value is set to 'false').
- method_request_parameters (map) A key-value map defining required or optional method request parameters that can be accepted by API Gateway. A key defines a method request parameter name matching the pattern method.request.{location}.{name}, where location is query string, path, or header and name is a valid and unique parameter name. The value associated with the key is a Boolean flag indicating whether the parameter is required (true) or optional (false)- not required (is not set).
- integration_request_body_template (map) Represents a map of Velocity templates that are applied on the request payload based on the of the Content-Type header sent by the client (is not set).
- responses (list) Method responses (sets default response with '200' status code).
- integration_responses (list) Integration method responses (sets the default response with '200' status code and without Lambda regex).
- default_error_pattern (list) Not required (if you did not specify integration_responses and responses, you can choose default).
- method_request_models (map) Specifies the Model resources used for the request's content type - not required (is not set).
- request_validator (map) A key-value map for creating request validator. If no name is specified, then will be used standard names.
- integration_type (string) [REQUIRED] The resource to which the method is connected.
- uri (string) Uniform Resource Identifier (URI) of the integration endpoint. Required if integration type is service.
- lambda_name (string) [REQUIRED] Lambda name. Required if integration type is lambda.
- enable_proxy (boolean) Present if only integration_type is Lambda.
- lambda_region (string) The Region, which value you can override from m3config.conf. Required (if Lambda is not in the same region as API).
- cache_configuration (map) The method cache configuration
- throttling_configuration The method throttling configuration
Precondition: The API Gateway resources and methods should be specified for the API Gateway deployment
Linked resources: api_gateway_resource: 4.5.2 API Gateway Resource api_gateway_resource_method: 4.5.3 API Gateway Resource method
"api_name": {
"deploy_stage": "dev",
"resource_type": "api_gateway,
"cluster_cache_configuration": {
"policy_statement_singleton": true|false,
"cache_enabled": true|false,
"cache_size": float,
"cache_ttl_sec": int,
"encrypt_cache_data": true|false
},
"cluster_throttling_configuration": {
"throttling_enabled": true|false,
"throttling_rate_limit": float,
"throttling_burst_limit": int
},
"dependencies": [
{
"resource_name": "lambda_name",
"resource_type": "lambda"
}
...
],
"resources": {
"/path": {
"enable_cors": true,
"POST|GET|DELETE|PUT|HEAD|PATCH|ANY":{
"authorization_type":" AWS_IAM|CUSTOM|COGNITO_USER_POOLS",
"api_key_required": true|false,
"method_request_parameters": {
"method.request.querystring.param_name": true|false
},
"method_request_models":{
"string": "string"
...
},
"cache_configuration": {
"cache_ttl_sec": int,
"encrypt_cache_data": true|false
},
"throttling_configuration": {
"throttling_enabled": true|false,
"throttling_rate_limit": float,
"throttling_burst_limit": int
}
"request_validator": {
"name": "string",
"validate_request_body": true|false,
"validate_request_parameters": true|false
}
"integration_type": "lambda|service|mock|http",
"uri": "region:subdomain.service|service:path|action/service_api",
"lambda_name": "name",
"enable_proxy": true|false,
"integration_request_body_template": {
"application/json": "..",
}
"integration_request_parameters": {
"integration.request.path|querystring.name": "method.request.path|querystring.param_name",
"integration.request.header.name": "..."
},
"integration_passthrough_behavior": "WHEN_NO_MATCH|WHEN_NO_TEMPLATES| NEVER"
"lambda_region": "one_of_the_aws_region",
"responses": [
{
"status_code": "status_code"
"response_parameters": {
"string": "string",
...
}
"response_models": {
"string": "string"
...
}
},
...
]
integration_responses: [
{
"status_code": "status_code",
"lambda_error_regex": "..",
"response_parameters": {
"string": "string",
...
}
"response_templates": {
"string": "string",
...
}
},
...
],
}
}
}
}
NOTE: Here we have an API Gateway description. This resource can be described in different deployment_resources.json files, part of API can be in one file, and another part - in another file. The 'resources' field can include not limited amount of resource paths.
Example:
"syndicate-demo-api": {
"resource_type": "api_gateway",
"deploy_stage": "dev",
"cluster_cache_configuration": {
"policy_statement_singleton": true,
"cache_enabled": true,
"cache_size": 0.5,
"cache_ttl_sec": 300,
"encrypt_cache_data": true
},
"cluster_throttling_configuration": {
"throttling_enabled": true,
"throttling_rate_limit": 10000,
"throttling_burst_limit": 5000
},
"dependencies": [
{
"resource_name": "put_dynamodb_item",
"resource_type": "lambda"
}
],
"resources": {
"/notifications": {
"enable_cors": true,
"POST": {
"integration_request_body_template": {},
"authorization_type": "AWS_IAM",
"integration_type": "lambda",
"method_request_parameters": {},
"cache_configuration": {
"cache_ttl_sec": 100,
"encrypt_cache_data": true
},
"throttling_configuration": {
"throttling_enabled": false,
"throttling_rate_limit": 10002,
"throttling_burst_limit": 5002
}
"default_error_pattern": true,
"integration_passthrough_behavior": "WHEN_NO_TEMPLATES",
"lambda_name": "put_dynamodb_item"
}
}
}
}
Example of Cognito UserPool usage as an authorizer:
"syndicate-demo-api": {
"resource_type": "api_gateway",
"deploy_stage": "dev",
"authorizers": {
"authorizer": {
"type": "COGNITO_USER_POOLS",
"identity_source": "method.request.header.Authorization",
"user_pools": [
"cognito_userpool_name"
],
"ttl": 300
}
},
"resources": {
"/notifications": {
"enable_cors": false,
"GET": {
"enable_proxy": true,
"authorization_type": "authorizer",
"integration_type": "lambda",
"lambda_name": "lambda_name",
"api_key_required": false,
"method_request_parameters": {},
"integration_request_body_template": {},
"responses": [],
"integration_responses": [],
"default_error_pattern": true
}
}
}
}
Example of Lambda function usage as an authorizer:
"syndicate-demo-api": {
"resource_type": "api_gateway",
"deploy_stage": "dev",
"dependencies": [],
"authorizers": {
"authorizer": {
"type": "REQUEST",
"identity_source": "method.request.querystring.principal_id, method.request.querystring.authorization_token, context.httpMethod, context.resourcePath",
"lambda_alias": "${lambdas_alias_name}",
"lambda_name": "lambda_authorizer_name",
"ttl": 300
}
},
"resources": {
"/notifications": {
"enable_cors": true,
"GET": {
"integration_type": "lambda",
"lambda_alias": "${lambdas_alias_name}",
"authorization_type": "authorizer",
"integration_request_body_template": {
"application/json": "#set($allParams = $input.params()){\"method\": \"$context.httpMethod\", \"path\": \"$context.resourcePath\",\"params\": {#foreach($type in $allParams.querystring.keySet())\"$type\": \"$util.escapeJavaScript($allParams.querystring.get($type))\" #if($foreach.hasNext),#end #end}, \"body_json\": $input.json('$')}"
},
"lambda_name": "booking-api"
}
}
}
}
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀Ways to deploy API Gateway to AWS Account with Syndicate
| ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ | ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ |
|---|---|
| Syndicate | OAS |
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀CREATE⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
| ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ | ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ |
|---|---|
| Create using Syndicate DSL (deployment resources) | Create using OAS v3 file (Composed manually or via AOSv3 Builders) |
| PLEASE NOTE: API Gateway extentions for OPEN API are required in order to configure AWS specific authentication and integration via OAS |
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀UPDATE
| ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ | ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ |
|---|---|
|
WARNING: API Gateway can't be updated via Syndicate. The API will be deleted and created from scratch. This may break integrations with API. To redeploy API: update API Gateway meta in Syndicate Deployment Resources syndicate clean syndicate build syndicate deploy
|
syndicate export (once only) update API Gateway meta in OAS file syndicate build syndicate update PLEASE NOTE: The API will be updated with no recreation. |
⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀CLEAN
| ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ | ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ |
|---|---|
syndicate clean |
syndicate clean |
| ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ |
|---|
| Recommended way: |
| We recommend deploying API Gateway via Syndicate to simplify all integrations with Lambdas and Authorizers. Then, export the OAS file using the syndicate export command in order to provide rolling update for your API Gateway |
| API IMPORT DOC |
| API IMPORT IMPORTANT NOTES |
INITIAL API DEFINING IN A PROJECT & TRANSITION TO OPEN API SPECIFICATION v3
Prerequisites: project is created, syndicate config is generated. 1. Generate Syndicate DSL of API Gateway
syndicate generate meta --resource_type api_gateway
syndicate generate meta --resource_type api_gateway_authorizer
syndicate generate meta --resource_type api_gateway_resource
syndicate generate meta --resource_type api_gateway_resource_method
2. Build Deployment Bundle
syndicate build
3. Deploy Bundle to AWS
syndicate deploy
4. Export OpenApi Specification
syndicate export --resource_type api_gateway --dsl oas_v3
NOTE: AWS API Gateway specific extensions are used to define the API in OAS v3, starting with "x-amazon"
5. Update Syndicate's Deployment Resources:
5.1. Remove the resource of type "api_gateway" from the project deployment_resources.json file
5.2. Add the resource of type "api_gateway_oas_v3" by placing the specification file inside of the project directory. The name of the file must end with oas_v3.json
5. Update the OAS File according to the project needs
6. Build Bundle with Updates
syndicate build
7. Update Environment including API via OAS
syndicate update
Congrats, you have migrated the API definition to OAS v3!
DEPLOYING NEW ENVIRONMENT CONTAINING API DEFINITION OASv3
1. Update the OASv3 file:
1.1. In case you have Cognito - API Gateway integration in your software the following security schema must be defined in your OAS document:
{
"securitySchemes": {
"authorizer": {
"type": "apiKey",
"name": "Authorization",
"in": "header",
"x-amazon-apigateway-authtype": "cognito_user_pools",
"x-amazon-apigateway-authorizer": {
"providerARNs": ["arn:aws:cognito-idp:$region:$account_id:userpool/$user_pool_id"],
"type": "cognito_user_pools"
}
}
}
}
Please, take a look at the 'x-amazon-apigateway-authorizer' object and it's 'providerARNS' property - it contains the ARN of the target Cognito UserPool.
As you are provisioning the new environment, we consider the Cognito UserPool does not exist yet and we don't know the actual ARN. The identifier of the User Pool is the combination of the deployment region and a unique ID of the pool. That is why the ARN can't be generated before it is created.
In case the Cognito UserPool is also defined in the Syndicate's deployment resources file, we recommend to replace the 'providerARNs' property with the following one:
"x-syndicate-cognito-userpool-names": ["cognito_userpool_name"]
Here, we're setting up the security rules for the API Gateway when we deploy it from scratch, using an OAS file that Syndicate exported from another environment:
"securitySchemes": {
"authorizer": {
"type": "apiKey",
"name": "Authorization",
"in": "header",
"x-amazon-apigateway-authtype": "cognito_user_pools",
"x-amazon-apigateway-authorizer": {
"x-syndicate-cognito-userpool-names": ["cognito_userpool_name"],
"type": "cognito_user_pools"
}
}
} During the 'syndicate deploy' command execution the syndicate will create resources in configured AWS Account according to defined priorities: Cognito UserPool will be created before the API Gateway. This allows Syndicate to create the Cognito UserPool, obtain it's ARN and replace the 'x-syndicate-cognito-userpool-names' property with the expected 'providerARNs' and the actual ARN of the UserPool referenced in the value.
1.2 Build the bundle
syndicate build
1.3 Deploy the app
syndicate deploy
Resource type: SNS Topic
Command to generate meta:
syndicate generate meta sns_topic
--resource_name demo-sns-topic
--region eu-central-1
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-sns-topic": {
"resource_type": "sns_topic",
"region": "eu-central-1",
"event_sources": []
}
}
Parameters:
"topic_name": {
"resource_type": "sns_topic" * Resource type. Required.
"region": /"all"/"region_name"/["region_name1", ..]
"event_sources": [
{
"target_rule": "rule_name",
"resource_type": "cloudwatch_rule_trigger"
}
]
}
- resource_type (string) [REQUIRED] Resource type.
- region (string) The region where the rule is deployed. If not specified ["region_name1", ..] the default value is taken from sdct.conf.
Example:
"stackAuditTopic": {
"region": "all",
"resource_type": "sns_topic"
}
Resource type: CloudWatch Alarms
Command to generate meta:
syndicate generate meta cloudwatch_alarm
--resource_name demo-cloudwatch-alarm
--metric_name demo-metric
--namespace demo-namespace
--period 1200
--evaluation_periods 1
--threshold 1.0
--comparison_operator GreaterThanOrEqualToThreshold
--statistic SampleCount
--sns_topics alarm-sns-topic
--lambdas audit-processor:5
--description "Custom SNS alarm"
--datapoints 1
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-cloudwatch-alarm": {
"resource_type": "cloudwatch_alarm",
"metric_name": "demo-metric",
"namespace": "demo-namespace",
"period": 1200,
"evaluation_periods": 1,
"threshold": 1.0,
"comparison_operator": "GreaterThanOrEqualToThreshold",
"statistic": "SampleCount",
"sns_topics": [
"alarm-sns-topic"
],
"lambdas": [
"audit-processor:5"
],
"description": "Custom SNS alarm"
}
}
Parameters:
"resource_type": "cloudwatch_alarm"
"alarm_name": {
"metric_name": "string",
"resource_type": "cloudwatch_alarm",
"period": 1200,
"evaluation_periods": 1,
"threshold": 1.0,
"comparison_operator": "GreaterThanOrEqualToThreshold|GreaterThanThreshold|LessThanThreshold|LessThanOrEqualToThreshold|LessThanLowerOrGreaterThanUpperThreshold|LessThanLowerThreshold|GreaterThanUpperThreshold",
"statistic": "SampleCount|Average|Sum|Minimum|Maximum",
"sns_topics": ["topic_name"],
"lambdas": ["lambda_name"]
"description": "string",
"evaluate_low_sample_count_percentile": "evaluate"|"ignore",
"datapoints": 1
}
- metric_name (string) [REQUIRED] The metric name.
- resource_type (string) [REQUIRED] Resource type.
- namespace (string) [REQUIRED] The namespace for the metric associated with the alarm.
- period (int) [REQUIRED] The period, in seconds, over which the specified statistic is applied. Valid values are 10, 30, and any multiple of 60.
- evaluation_periods (int) [REQUIRED] A number of periods over which data is compared to the specified threshold.
- treshold (float) [REQUIRED] The value to compare with the specified statistic.
- comparison_operator (string) [REQUIRED] An arithmetic operation to use when comparing the specified statistic and threshold. The specified statistic value is us as the first operand.
- statistic (string) [REQUIRED] The statistic for the metric associated with the alarm, other than percentile. For percentile statistics, use ExtendedStatistic.
- sns_topics (list) The actions to execute when this alarm transitions to an ALARM state from any other state. Each action is specified as a name of SNS topics.
-
lambdas (list)
The actions to execute when this alarm transitions to an ALARM state from any other state. Each action is specified as a name of lambda. Use
:after lambda name to specify alias or version. - ssm_response_plan (list) The actions to execute when this alarm transitions to an ALARM state from any other state. Each action is specified as a name of response plan.
- description (string) The description for the alarm.
- evaluate_low_sample_count_percentile (string) Only for percentiles-based alarms. Use 'ignore' and the alarm state remains unchanged during periods with insufficient data points for statistical significance. If 'evaluate' is specified (or parameter is omitted), the alarm is always assessed and may change state regardless of data point availability.
-
datapoints (int)
The number of datapoints that must be breaching to trigger the alarm. Must be lower or equal to the
evaluation_periods.
Example:
"alarm_name": {
"metric_name": "db_alarm",
"resource_type": "cloudwatch_alarm"
"namespace": "db",
"period": 1200,
"evaluation_periods": 1
"threshold": 1.0,
"comparison_operator": "GreaterThanOrEqualToThreshold",
"statistic": "SampleCount",
"sns_topics": [
"audit_topic"
]
}
Resource type: Kinesis Stream
Command to generate meta:
syndicate generate meta kinesis_stream
--resource_name demo-kinesis-stream
--shard_count 3
JSON Meta generated by the CLI command (deployment_resources.json):
{
" demo-kinesis-stream ": {
"resource_type": "kinesis_stream",
"shard_count": 3
}
}
Parameters:
"stream_name": {
"resource_type": "kinesis_stream",
"shard_count": 2
}
- resource_type (string) [REQUIRED] Resource type.
- shard_count (int) [REQUIRED] Number of shards that the stream uses.
Example:
"audit_stream": {
"resource_type": "kinesis_stream",
"shard_count": 1
}
Resource type: IAM Policy
Command to generate meta:
syndicate generate meta iam_policy
--resource_name demo-lambda-execution-policy
--policy_content policy.json
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-lambda-execution-policy": {
"resource_type": "iam_policy",
"policy_content": {
"Statement": [
{
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Effect": "Allow",
"Resource": "*"
}
],
"Version": "2012-10-17"
}
}
}
Precondition:
file policy.json should be previously created and put into the current directory. Example of file content:
{
"Statement": [
{
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Effect": "Allow",
"Resource": "*"
}
],
"Version": "2012-10-17"
}
Parameters:
“policy_name”: {
"resource_type": "iam_policy",
"policy_content": {
...
}
}
- resource_type (string) [REQUIRED] Resource type.
- policy_content (map) [REQUIRED] IAM policy content
Example:
"AutoscalingDynamoRead": {
"resource_type": "iam_policy",
"policy_content": {
"Version": "2012- 10 - 17",
"Statement": [
{
"Action": [
"dynamodb:DescribeTable",
"cloudwatch:GetMetricStatistics",
"cloudwatch:DescribeAlarms"
],
"Resource": "*",
"Effect": "Allow"
}
]
}
}
Resource type: Iam Role
Command to generate meta:
syndicate generate meta iam_role
--resource_name demo-lambda-role
--principal_service lambda
--predefined_policies AmazonS3ReadOnlyAccess
--custom_policies demo-lambda-execution-policy
--allowed_accounts 559465876067
--external_id 559465876067
--instance_profile false
--permissions_boundary arn:aws:iam::559465876067:policy/DemoPermissionBoundaries
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-lambda-execution-policy": {
"resource_type": "iam_policy",
"policy_content": {
"Statement": [
{
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Effect": "Allow",
"Resource": "*"
}
],
"Version": "2012-10-17"
}
},
"demo-lambda-role": {
"resource_type": "iam_role",
"principal_service": "lambda",
"predefined_policies": [
"AmazonS3ReadOnlyAccess"
],
"custom_policies": [
"demo-lambda-execution-policy"
],
"allowed_accounts": [
"${account_id}"
],
"external_id": "859465876068",
"instance_profile": false,
"permissions_boundary": "arn:aws:iam::${account_id}:policy/DemoPermissionBoundaries"
}
}
Linked resources: custom_policies : 4.9 IAM Policy
Parameters:
"role_name": {
"predefined_policies": [ - Managed IAM policies list.
"policy_name"
],
"principal_service": "lambda", - Service which uses the role.
"custom_policies": [
"LambdaBasicExecution",
"S3Read",
"SNSWrite",
"CloudFormationResourceCreationPolicyWrite"
],
"resource_type": "iam_role", * Resource type. Required.
"allowed_accounts": [ - The list of accounts, which can assume the role.
"123456789123"
],
" **external_id** ": "your_id", - External ID in role.
" **instance_profile** ": true|false, - If true, instance profile with role name is created.
" **trusted_relationships** ": {} - The .json-file of the trusted relationships to be attached.
}
- resource_type (string) [REQUIRED] Resource type.
- predefined_policies (list) Managed IAM policies list.
- principal_service (string) Service which uses the role.
- custom_policies (list) Customer AWS policies names.
- allowed_accounts (list) The list of accounts, which can assume the role.
- external_id (string) External ID in role.
- instance_profile (boolean) If true, instance profile with role name is created.
- trusted_relationships (map) The .json-file of the trusted relationships to be attached.
Example:
"lr_run_terraform_template": {
"predefined_policies": [
"AmazonSQSFullAccess"
],
"principal_service": "lambda",
"custom_policies": [
"LambdaBasicExecution",
"S3Read",
"SNSWrite",
"CloudFormationResourceCreationPolicyWrite"
],
"resource_type": "iam_role",
"allowed_accounts": [
"${account_id}"
]
}
Resource type: Step Function Activity
Command to generate meta:
syndicate generate meta step_function_activity
--resource_name demo-activity
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-activity": {
"resource_type": "state_activity"
}
}
Parameters:
"activity_name": {
"resource_type": "state_activity", * Resource type. Required.
}
- resource_type (string) [REQUIRED] Resource type.
Example:
"approval_activity": {
"resource_type": "state_activity"
}
Resource type: SQS Queue
Command to generate meta:
syndicate generate meta sqs_queue
--resource_name demo-sqs-queue
--region eu-central-1
--fifo_queue false
--visibility_timeout 30
--delay_seconds 0
--maximum_message_size 1024
--message_retention_period 60
--receive_message_wait_time_seconds 20
--dead_letter_target_arn arn:aws:sqs:eu-central-1:859465876068:dead-letter-queue
--max_receive_count 2
--kms_master_key_id b328b33f-51fd-44ab-bdd9-e182a015bffd
--kms_data_key_reuse_period_seconds 60
--content_based_deduplication false
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-sqs-queue": {
"resource_type": "sqs_queue",
"fifo_queue": false,
"region": "eu-central-1",
"visibility_timeout": 30,
"delay_seconds": 0,
"maximum_message_size": 1024,
"message_retention_period": 60,
"receive_message_wait_time_seconds": 20,
"policy": {},
"redrive_policy": {
"deadLetterTargetArn": "arn:aws:sqs:eu-central-1:859465876068:dead-letter-queue",
"maxReceiveCount": 2
},
"kms_master_key_id": "b328b33f-51fd-44ab-bdd9-e182a015bffd",
"kms_data_key_reuse_period_seconds": 60,
"content_based_deduplication": false
}
}
Parameters:
"audit-queue-name": {
"region": "string",
"fifo_queue": boolean,
"visibility_timeout": int,
"resource_type": "string",
"delay_seconds": int,
"maximum_message_size": int,
"message_retention_period": int,
"policy": {},
"receive_message_wait_time_seconds": int,
"redrive_policy": {
"deadLetterTargetArn": "string",
"maxReceiveCount": int
},
"kms_master_key_id": "string",
"kms_data_key_reuse_period_seconds": int,
"content_based_deduplication": boolean
}
- region (string) The region, where the queue is deployed (the default value is the region from sdct.conf).
- fifo_queue (boolean) If true, the queue is FIFO (the default value is false).
- visibility_timeout (int) The visibility timeout for the queue.
- resource_type (string) [REQUIRED] Resource type.
- delay_seconds (int) The length of time, in seconds, for which the delivery of all messages in the queue is delayed.
- maximum_message_size (int) The limit of how many bytes a message can contain before Amazon SQS rejects it.
- message_retention_period (int) The length of time, in seconds, for which Amazon SQS retains a message.
- policy (json) The queue's policy. A valid AWS policy.
- receive_message_wait_time_seconds (string) The length of time, in seconds, for which a "ReceiveMessage" action waits for a message to arrive.
-
redrive_policy (json)
Not required.
- deadLetterTargetArn (string) [REQUIRED] The Amazon Resource Name (ARN) of the "arn", dead-letter queue to which Amazon SQS moves messages after the value of maxReceiveCount is exceeded.
- maxReceiveCount (int) [REQUIRED] The number of times a message is delivered to the source queue before being moved to the dead-letter queue.
- kms_master_key_id (string) The ID of an AWS-managed customer "alias/aws/sqs", master key (CMK) for Amazon SQS or a custom CMK.
- kms_data_key_reuse_period_seconds (int) The length of time, in seconds, for which Amazon SQS can reuse a data key to encrypt or decrypt messages before calling AWS KMS again.
- content_based_deduplication (boolean) Enables content-based.
Example:
"${terraform-queue-name}": {
"region": "eu-west-1",
"fifo_queue": true,
"visibility_timeout": 300,
"resource_type": "sqs_queue"
}
Resource type: Step Functions (State machine)
Command to generate meta:
syndicate generate meta step_function
--resource_name demo-step-function
--iam_role StepFunctionsLambdaRole
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-step-function": {
"resource_type": "step_functions",
"definition": {
"States": {
"ExampleState": {
"Type": "Succeed"
}
},
"Comment": "A description of your state machine",
"StartAt": "ExampleState"
},
"event_sources": [],
"dependencies": [],
"iam_role": "StepFunctionsLambdaRole"
}
}
Linked resources: iam_role: 4.10 IAM Role
Useful links: Follow this guide to create an IAM role for your state machine
Parameters:
"resource_type": "step_functions" "resource_type": "cloudwatch_rule_trigger"
"state_machine_collect_reports": {
"definition": {
…
},
"iam_role": "string",
"resource_type": "string",
"event_sources": [
{
"input": {
"event": "string"
},
"iam_role": "string",
"resource_type": "string",
"target_rule": "string"
}
]
}
- definition [REQUIRED] The Amazon States Language definition of the state machine.
- iam_role (string) [REQUIRED] IAM role to use for this state machine.
- resource_type (string) [REQUIRED] Resource type.
- event_sources Subscriptions.
- input [REQUIRED] Input to Cloudwatch rule.
- iam_role (string) [REQUIRED] IAM role name to use for this state machine.
- resource_type (string) [REQUIRED] Resource type.
- target_rule (string) [REQUIRED] Name of the CloudWatch rule.
Example:
"state_machine_collect_nessus_reports": {
"definition": {
"States": {
"GoogleMatchState": {
"InputPath": "$.lambdaPayload",
"End": true,
"Type": "Task",
"Lambda": "lambda_collect_google_nessus_reports"
},
"ChoiceState": {
"Default": "DefaultState",
"Type": "Choice",
"Choices": [
{
"Variable": "$.cloud",
"StringEquals": "AWS",
"Next": "AwsMatchState"
},
{
"Variable": "$.cloud",
"StringEquals": "Google",
"Next": "GoogleMatchState"
}
]
},
"AwsMatchState": {
"InputPath": "$.lambdaPayload",
"End": true,
"Type": "Task",
"Lambda": "lambda_collect_aws_nessus_reports"
},
"DefaultState": {
"Cause": "No Matches!",
"Type": "Fail"
},
"WaitState": {
"SecondsPath": "$.timeToWaitInSeconds",
"Type": "Wait",
"Next": "ChoiceState"
}
},
"StartAt": "WaitState"
},
"dependencies": [
{
"resource_name": "lambda_collect_aws_nessus_reports",
"resource_type": "lambda"
},
{
"resource_name": "lambda_collect_google_nessus_reports",
"resource_type": "lambda"
}
],
"iam_role": "state_machine_role",
"resource_type": "step_functions"
}
NOTE: If the field 'Lambda' is present, Lambda function is attached to the state (in the same way the Activity field can be represented).
Resource type: Cognito User Pools
Command to generate meta:
syndicate generate meta cognito_user_pool
--resource_name demo-user-pool
--auto_verified_attributes email
--auto_verified_attributes phone_number
--sns_caller_arn arn:aws:iam::123412341234:role/CognitoSMSRole
--username_attributes email
--username_attributes phone_number
--custom_attributes fullname String
--custom_attributes birthday DateTime
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-user-pool": {
"resource_type": "cognito_idp",
"password_policy": {
"minimum_length": 8,
"require_uppercase": true,
"require_symbols": true,
"require_lowercase": true,
"require_numbers": true
},
"auto_verified_attributes": [
"email",
"phone_number"
],
"sms_configuration": {
"sns_caller_arn": "arn:aws:iam::123412341234:role/CognitoSMSRole"
},
"username_attributes": [
"email",
"phone_number"
],
"custom_attributes": [
{
"name": "fullname",
"type": "String"
},
{
"name": "birthday",
"type": "DateTime"
}
],
"client": {}
}
}
Preconditions:
- CognitoSMSRole should be set in the account.
Linked resources:
- iam_role: 4.10 IAM Role
Useful links:
- Follow this guide to prepare an IAM role that Amazon Cognito can use to send SMS messages with Amazon SNS
Resource type: Amazon Cognito Identity pools
Command to generate meta:
syndicate generate meta cognito_federated_pool
--resource_name demo-identity-pool
--auth_role DemoUserRole
--unauth_role DemoGusetRole
--open_id_providers accounts.google.com
--provider_name demo-developer-provider
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-federation-pool": {
"resource_type": "cognito_federated_pool",
"auth_role": "DemoUserRole",
"unauth_role": "DemoGusetRole",
"open_id_providers": [
"accounts.google.com"
],
"provider_name": "demo-developer-provider"
}
}
Preconditions:
- OpenID Connect provider 'accounts.google.com' should be previously defined in account.
Linked resources:
- iam_role: 4.10 IAM Role
Useful links:
- Follow this guide to create and manage IAM OIDC identity providers.
Parameters:
"resource_type": "cognito_federated_pool"
"maestro3_epam_opensource": {
"auth_role": "string",
"unauth_role": "string",
"open_id_providers": [
"string"
],
"provider_name": "string",
"resource_type": "string"
}
- auth_role (string) IAM role for authorized users.
- unauth_role (string) IAM role for unauthorized users.
- open_id_providers (list) A list of OpendID Connect providers.
- provider_name (string) Developer provider name
- resource_type [REQUIRED] Resource type. Required.
Example:
"maestro3_epam_opensource": {
"auth_role": "cognito_auth",
"open_id_providers": [
"accounts.google.com"
],
"provider_name": "login.m3.com",
"resource_type": "cognito_federated_pool"
}
Resource type: SNS Application
Command to generate meta:
syndicate generate meta sns_application
Sample for GCM:
syndicate generate meta sns_application
--resource_name demo-sns-application
--platform GCM
--region eu-central-1
--attributes PlatformCredential ${google_api_key}
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-sns-application": {
"resource_type": "sns_application",
"platform": "GCM",
"region": "eu-central-1",
"attributes": {
"PlatformCredential": "${google_api_key} "
}
}
}
Preconditions:
- GCM (FCM) API key for PlatformCredential should be previously obtained from Google and specified in the syndicate_aliases.yml file as the google_api_key property
Parameters:
"mobile-app": {
"platform":"string",
"region": "string",
"resource_type": "string",
"attributes": {
"attr_name": "attr_value"
…
}
}
- platform (string) [REQUIRED] SNS application platform.
- region (string) Regions name/list, where the application is deployed (if not stated, is deployed only in the region).
- resource_type (string) [REQUIRED] Resource type.
- attributes (string) [REQUIRED] SNS application attributes.
Example:
"mobile-app": {
"platform": "GCM",
"region": "eu-central-1",
"resource_type": "sns_application",
"attributes": {
"PlatformCredential": "${google_api_key}"
}
}
"resource_type": "beanstalk_app"
"aws_csv_billing": {
"resource_type": "string"
"deployment_package": "string",
"env_name": "string",
"notification_topic": "string",
"ec2_key_pair": "string",
"ec2_role": "string",
"ebs_service_role": "string",
"tier": {
"Name": "string",
"Type": "string"
},
"stack": "string",
"env_settings": [
{
"OptionName": "string",
"ResourceName": "string",
"Namespace": "string",
"Value": "string"
}
…
]
}
Parameters:
- resource_type (string) [REQUIRED] Resource type.
- deployment_package (string) [REQUIRED] Application artifact name.
- env_name (string) EBS environment name.
- notification_topic (string) SNS topic name to configure "ebs_notification", notifications.
- ec2_key_pair (string) [REQUIRED] EC2 key to run an instance.
- ec2_role (string) [REQUIRED] EC2 instance role.
- ebs_service_role (string) [REQUIRED] EBS service role.
- tier (string) [REQUIRED] EBS tier.
- stack (string) [REQUIRED] EBS stack.
- env_settings (string) If specified, AWS Elastic Beanstalk sets the specified configuration options to the requested value in the configuration set for the new environment.
Example:
{
"aws_csv_billing": {
"resource_type": "beanstalk_app",
"deployment_package": "m3-server-1.0.0.war",
"env_name": "m3-billing-env",
"notification_topic": "ebs_notification",
"ec2_key_pair": "m3_deployment",
"ec2_role": "ebs_instance_role",
"ebs_service_role": "ebs_service_role",
"tier": {
"Name": "WebServer",
"Type": "Standard"
},
"stack": "64bit Amazon Linux 2017.03 v2.6.3 running Tomcat 8 Java 8",
"env_settings": [
{
"OptionName": "Availability Zones",
"ResourceName": "AWSEBAutoScalingGroup",
"Namespace": "aws:autoscaling:asg",
"Value": "Any"
},
{
"OptionName": "Cooldown",
"ResourceName": "AWSEBAutoScalingGroup",
"Namespace": "aws:autoscaling:asg",
"Value": "360"
},
{
"OptionName": "MaxSize",
"ResourceName": "AWSEBAutoScalingGroup",
"Namespace": "aws:autoscaling:asg",
"Value": "1"
},
{
"OptionName": "MinSize",
"ResourceName": "AWSEBAutoScalingGroup",
"Namespace": "aws:autoscaling:asg",
"Value": "1"
},
{
"OptionName": "ImageId",
"ResourceName": "AWSEBAutoScalingLaunchConfiguration",
"Namespace": "aws:autoscaling:launchconfiguration",
"Value": "ami-ebd02392"
},
{
"OptionName": "InstanceType",
"Namespace": "aws:autoscaling:launchconfiguration",
"Value": "t2.micro"
},
{
"OptionName": "MonitoringInterval",
"ResourceName": "AWSEBAutoScalingLaunchConfiguration",
"Namespace": "aws:autoscaling:launchconfiguration",
"Value": "5 minute"
},
{
"OptionName": "RollingUpdateEnabled",
"ResourceName": "AWSEBAutoScalingGroup",
"Namespace": "aws:autoscaling:updatepolicy:rollingupdate",
"Value": "false"
},
{
"OptionName": "RollingUpdateType",
"ResourceName": "AWSEBAutoScalingGroup",
"Namespace": "aws:autoscaling:updatepolicy:rollingupdate",
"Value": "Time"
},
{
"OptionName": "HooksPkgUrl",
"Namespace": "aws:cloudformation:template:parameter",
"Value": "https://s3-eu-west-1.amazonaws.com/elasticbeanstalk-envresources-eu-west-1/stalks/eb_tomcat_4.0.1.148.17/lib/hooks.tar.gz"
},
{
"OptionName": "InstancePort",
"Namespace": "aws:cloudformation:template:parameter",
"Value": "80"
},
{
"OptionName": "JVMOptions",
"Namespace": "aws:cloudformation:template:parameter",
"Value": "XX:MaxPermSize=64m,Xmx=256m,JVM Options=,Xms=256m"
},
{
"OptionName": "Application Healthcheck URL",
"Namespace": "aws:elasticbeanstalk:application",
"Value": ""
},
{
"OptionName": "DeleteOnTerminate",
"Namespace": "aws:elasticbeanstalk:cloudwatch:logs",
"Value": "false"
},
{
"OptionName": "RetentionInDays",
"Namespace": "aws:elasticbeanstalk:cloudwatch:logs",
"Value": "7"
},
{
"OptionName": "StreamLogs",
"Namespace": "aws:elasticbeanstalk:cloudwatch:logs",
"Value": "false"
},
{
"OptionName": "BatchSize",
"Namespace": "aws:elasticbeanstalk:command",
"Value": "100"
},
{
"OptionName": "BatchSizeType",
"Namespace": "aws:elasticbeanstalk:command",
"Value": "Percentage"
},
{
"OptionName": "IgnoreHealthCheck",
"Namespace": "aws:elasticbeanstalk:command",
"Value": "false"
},
{
"OptionName": "Timeout",
"Namespace": "aws:elasticbeanstalk:command",
"Value": "600"
},
{
"OptionName": "JVM Options",
"Namespace": "aws:elasticbeanstalk:container:tomcat:jvmoptions",
"Value": ""
},
{
"OptionName": "XX:MaxPermSize",
"Namespace": "aws:elasticbeanstalk:container:tomcat:jvmoptions",
"Value": "64m"
},
{
"OptionName": "Xms",
"Namespace": "aws:elasticbeanstalk:container:tomcat:jvmoptions",
"Value": "256m"
},
{
"OptionName": "Xmx",
"Namespace": "aws:elasticbeanstalk:container:tomcat:jvmoptions",
"Value": "256m"
},
{
"OptionName": "DefaultSSHPort",
"Namespace": "aws:elasticbeanstalk:control",
"Value": "22"
},
{
"OptionName": "LaunchTimeout",
"Namespace": "aws:elasticbeanstalk:control",
"Value": "0"
},
{
"OptionName": "LaunchType",
"Namespace": "aws:elasticbeanstalk:control",
"Value": "Migration"
},
{
"OptionName": "RollbackLaunchOnFailure",
"Namespace": "aws:elasticbeanstalk:control",
"Value": "false"
},
{
"OptionName": "EnvironmentType",
"Namespace": "aws:elasticbeanstalk:environment",
"Value": "SingleInstance"
},
{
"OptionName": "GzipCompression",
"Namespace": "aws:elasticbeanstalk:environment:proxy",
"Value": "true"
},
{
"OptionName": "ProxyServer",
"Namespace": "aws:elasticbeanstalk:environment:proxy",
"Value": "apache"
},
{
"OptionName": "HealthCheckSuccessThreshold",
"Namespace": "aws:elasticbeanstalk:healthreporting:system",
"Value": "Ok"
},
{
"OptionName": "SystemType",
"Namespace": "aws:elasticbeanstalk:healthreporting:system",
"Value": "enhanced"
},
{
"OptionName": "LogPublicationControl",
"Namespace": "aws:elasticbeanstalk:hostmanager",
"Value": "false"
},
{
"OptionName": "ManagedActionsEnabled",
"Namespace": "aws:elasticbeanstalk:managedactions",
"Value": "false"
},
{
"OptionName": "InstanceRefreshEnabled",
"Namespace": "aws:elasticbeanstalk:managedactions:platformupdate",
"Value": "false"
},
{
"OptionName": "Automatically Terminate Unhealthy Instances",
"Namespace": "aws:elasticbeanstalk:monitoring",
"Value": "true"
},
{
"OptionName": "Notification Protocol",
"Namespace": "aws:elasticbeanstalk:sns:topics",
"Value": "email"
},
{
"OptionName": "XRayEnabled",
"Namespace": "aws:elasticbeanstalk:xray",
"Value": "false"
},
{
"OptionName": "EnvironmentVariables",
"Namespace": "aws:cloudformation:template:parameter",
"Value": "HOME_REGION="
},
{
"OptionName": "HOME_REGION",
"Namespace": "aws:elasticbeanstalk:application:environment",
"Value": "${billing_home_region}"
},
{
"OptionName": "EnvironmentVariables",
"Namespace": "aws:cloudformation:template:parameter",
"Value": "HOME_ACCOUNT_ID="
},
{
"OptionName": "HOME_ACCOUNT_ID",
"Namespace": "aws:elasticbeanstalk:application:environment",
"Value": "${billing_home_account_id}"
}
]
}
}
Resource type: EC2 Instance
Command to generate meta:
syndicate generate meta ec2_instance
--resource_name demo-instance
--key_name demo-key
--image_id ami-03cceb19496c25679
--instance_type t2.micro
--disable_api_termination false
--security_group_ids sg-0aea18793dd1fa3d9
--availability_zone eu-central-1a
--subnet_id subnet-0ab65ee0e036f0daa
--userdata_file demo-scrypt.sh
--iam_role DemoEC2Role
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-instance": {
"resource_type": "ec2_instance",
"key_name": "demo-key",
"image_id": "ami-03cceb19496c25679",
"instance_type": "t2.micro",
"availability_zone": "eu-central-1a",
"subnet_id": "subnet-0ab65ee0e036f0daa",
"userdata_file": "demo-scrypt.sh",
"disableApiTermination": false,
"iam_role": "DemoEC2Role",
"security_group_names": [],
"security_group_ids": [
"sg-0aea18793dd1fa3d9"
]
}
}
Preconditions:
- SSH key 'demo-key' should be previously set in account
- specified availability zone should be enabled
- security_group_ids should refer to existing group
- image_id should refer to an existing image
- the file 'demo-scrypt.sh' should be previously created.
Example of 'demo-scrypt.sh' file content:
#!/bin/bash yum update -y service httpd start chkconfig httpd onLinked resources:
- iam_role: 4.10 IAM Role
Useful links:
Parameters:
"admin-instance": {
"security_group_names": [
"string"
],
"security_group_ids": [
"string"
],
"availability_zone": "string",
"instance_type": "string",
"subnet_id": "string",
"key_name": "string",
"image_id": "string",
"userdata_file": "string",
"resource_type": "string",
"disableApiTermination": boolean,
"iam_role": "string"
}
- resource_type (string) [REQUIRED] Resource type.
- security_group_names (list) Security group names
- security_group_ids (list) Security group IDs.
- availability_zone (string) Availability zone.
- instance_type (string) [REQUIRED] Instance type.
- subnet_id (string) Subnet ID (needed if availability_zone is present).
- key_name (string) [REQUIRED] SSH key.
- image_id (string) [REQUIRED] Image ID.
- userdata_file (string) File path to userdata "instance_userdata.sh", (file relative pathname from the directory, which is set up in the environmental variable SDCT_CONF).
- disableApiTermination (string) API termination protection.
- iam_role (string) Instance IAM role. Example:
"instance": {
"security_group_ids": [
"${customer_sg_id}"
],
"instance_type": "t2.micro",
"subnet_id": "${subnet_id}",
"key_name": "${instance_key_name}",
"image_id": "${instance_image}",
"userdata_file": "instance_userdata.sh",
"resource_type": "ec2_instance",
"disableApiTermination": true,
"iam_role": "InstanceRole"
}
Resource type: EC2 Launch Template
Command to generate meta:
syndicate generate meta ec2_launch_template
--resource_name demo-launch_template
--key_name demo-key
--image_id ami-03cceb19496c25679
--instance_type t2.micro
--security_group_ids sg-0aea18793dd1fa3d9
--userdata_file demo-scrypt.sh
--iam_role DemoEC2Role
--imds_version v2.0
--version_description "version 1 for demo"
JSON Meta generated by the CLI command (deployment_resources.json):
"demo-launch_template": {
"resource_type": "ec2_launch_template",
"version_description": "version 1 for demo",
"launch_template_data": {
"key_name": "demo-key",
"image_id": "ami-03cceb19496c25679",
"instance_type": "t2.micro",
"security_group_ids": [
"sg-0aea18793dd1fa3d9"
],
"userdata_file": "demo-scrypt.sh",
"iam_role": "DemoEC2Role",
"imds_support": "v2.0"
}
}
Preconditions:
- SSH key 'demo-key' should be previously set in account
- security_group_ids should refer to existing group
- image_id should refer to an existing image
- the file 'demo-scrypt.sh' should be previously created.
- Example of 'demo-scrypt.sh' file content:
#!/bin/bash yum update -y service httpd start chkconfig httpd onLinked resources:
iam_role: 4.10 IAM Role
Useful links:
Parameters:
"demo-launch_template": {
"resource_type": "string",
"version_description": "version 1 for demo",
"launch_template_data": {
"key_name": "demo-key",
"image_id": "ami-03cceb19496c25679",
"instance_type": "t2.micro",
"security_group_ids": [
"sg-0aea18793dd1fa3d9"
],
"userdata_file": "demo-scrypt.sh",
"iam_role": "DemoEC2Role",
"imds_support": "v2.0"
}
}
- resource_type (string) [REQUIRED] Resource type.
- "version_description" (string) A description of the launch template version.
- "launch_template_data" (map) [REQUIRED] Resource type. Core information on the launch template configuration.
- "image_id" (string) [REQUIRED] Image ID.
- "security_groups" (list) Security group names.
- "security_group_ids" (list) Security group IDs.
- "instance_type" (string) Instance type.
- "key_name" (string) SSH key.
- "userdata_file" (string) File path to userdata "instance_userdata.sh", (can be specified as a relative path to the project path).
- "iam_role" (string) Instance IAM role.
- "imds_support" (string) ["v1.0", "v2.0"] The version of the IMDS.
Resource type: AWS Batch Compute environments
Command to generate meta:
syndicate generate meta batch_compenv
Sample for EC2 type:
syndicate generate meta batch_compenv
--resource_name demo-batch-compute-env
--compute_environment_type MANAGED
--state ENABLED
--service_role AWSBatchServiceRole
--type EC2
--minv_cpus 0
--maxv_cpus 2
--desiredv_cpus 1
--instance_types m3.medium
--security_group_ids sg-0aea18793dd1fa3d9
--subnets subnet-0ab65ee0e036f0daa
--subnets subnet-0189d0206149b0c36
--instance_role InstanceRole
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-batch-compute-env": {
"resource_type": "batch_compenv",
"compute_environment_type": "MANAGED",
"state": "ENABLED",
"service_role": "AWSBatchServiceRole",
"compute_resources": {
"type": "EC2",
"minv_cpus": 0,
"maxv_cpus": 2,
"desiredv_cpus": 1,
"instance_types": [
"m3.medium"
],
"security_group_ids": [
"sg-0aea18793dd1fa3d9"
],
"subnets": [
"subnet-0ab65ee0e036f0daa",
"subnet-0189d0206149b0c36"
],
"instance_role": "InstanceRole"
}
}
}
Sample for Fargate type:
syndicate generate meta batch_compenv
--resource_name demo-batch-compute-env
--compute_environment_type MANAGED
--state ENABLED
--type FARGATE
--maxv_cpus 2
--security_group_ids sg-0aea18793dd1fa3d9
--subnets subnet-0ab65ee0e036f0daa
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-batch-compute-env": {
"resource_type": "batch_compenv",
"compute_environment_type": "MANAGED",
"state": "ENABLED",
"compute_resources": {
"type": "FARGATE",
"maxv_cpus": 2,
"instance_types": [],
"security_group_ids": [
"sg-0aea18793dd1fa3d9"
],
"subnets": [
"subnet-0ab65ee0e036f0daa"
]
}
}
}
Linked resources: iam_role: 4.10 IAM Role
Parameters:
-
resource_type (string) [REQUIRED] Resource type.
-
compute_environment_type (string) The type of the compute environment:
MANAGEDorUNMANAGED. The default value isENABLED -
state (string) The state of the compute environment:
ENABLEDorDISABLED. The default value isENABLED -
service_role (string) The full Amazon Resource Name (ARN) of the IAM role that allows AWS Batch to make calls to other AWS services on your behalf. If not specified, syndicate will create 'AWSBatchServiceRole' with attached service-role policy: 'AWSBatchServiceRole'.
-
compute_resources (dict)
[REQUIRED] ifcompute_environment_typeset toMANAGEDDetails about the compute resources managed by the compute environment. Should be described as a dict of the following format:
{ "type": "EC2" | "SPOT" | "FARGATE" | "FARGATE_SPOT", "allocation_strategy" : "BEST_FIT" | "BEST_FIT_PROGRESSIVE" | "SPOT_CAPACITY_OPTIMIZED", "minv_cpus": 123, "maxv_cpus": 123, "desiredv_сpus": 123, "instance_types": [ "string", ], "image_id": "string", "subnets": [ "string", ], "security_group_ids": [ "string" ], "ec2_key_pair": "string", "instance_role": "string", "tags": { "string": "string" }, "placement_group": "string", "bid_percentage": 123, "spot_iam_fleet_role": "string", "launch_template":{ "launch_template_id": "string", "launch_template_name": "string", "version": "string" }, "ec2_configuration":[ { "image_type": "string", "image_id_override": "string" }, ] }-
type (string) [REQUIRED]
Should be one of available values: "EC2", "SPOT", "FARGATE", "FARGATE_SPOT"; -
allocation_strategy (string) Should be one of available values: "BEST_FIT", "BEST_FIT_PROGRESSIVE", "SPOT_CAPACITY_OPTIMIZED";
-
minv_cpus (int): The minimum number of Amazon EC2 vCPUs that an environment should maintain (even if the compute environment is DISABLED );
-
maxv_cpus (int) [REQUIRED] The maximum number of Amazon EC2 vCPUs that a compute environment can reach.
-
desiredv_cpus (int)
The desired number of Amazon EC2 vCPUS in the compute environment. AWS Batch modifies this value between the minimum and maximum values, based on job queue demand. -
instance_types (list) The instances types that can be launched. You can specify instance families to launch any instance type within those families (for example, c5 or p3 ), or you can specify specific sizes within a family (such as c5.8xlarge ). You can also choose optimal to select instance types (from the C4, M4, and R4 instance families) that match the demand of your job queues.
-
imageId (string)
The Amazon Machine Image (AMI) ID used for instances launched in the compute environment. This parameter is overridden by theimage_id_overridemember of theec2_configurationstructure. -
subnets (list) [REQUIRED] The VPC subnets into which the compute resources are launched. These subnets must be within the same VPC.
-
security_group_ids (list) The Amazon EC2 security groups associated with instances launched in the compute environment.
-
ec2_key_pair (string) The Amazon EC2 key pair that's used for instances launched in the compute environment. You can use this key pair to log in to your instances with SSH.
-
instance_role (string) The Amazon ECS instance profile applied to Amazon EC2 instances in a compute environment. You can specify the short name or full Amazon Resource Name (ARN) of an instance profile.
-
tags (dict) Key-value pair tags to be applied to EC2 resources that are launched in the compute environment.
-
Example compute environment:
{
"some_best_compenv": {
"resource_type": "batch_compenv",
"compute_environment_type": "MANAGED",
"state": "ENABLED",
"service_role": "AWSServiceRoleForBatch",
"compute_resources": {
"type": "EC2",
"minv_cpus": 1,
"maxv_cpus": 2,
"instance_types": [
"p3"
],
"security_group_ids": [
"sg-3f7da44f"
],
"subnets": [
"subnet-2ec5b544",
"subnet-00f35d7c",
"subnet-31bc017d"
],
"instance_role": "AmazonEC2ContainerServiceforEC2Role",
"tags": {
"name": "my_compenv"
}
}
}
}
Command to generate meta:
syndicate generate meta batch_jobqueue
--resource_name demo-batch-job-queue
--state ENABLED
--priority 1
--compute_environment_order 1 demo-batch-compute-env
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-batch-job-queue": {
"resource_type": "batch_jobqueue",
"state": "ENABLED",
"priority": 1,
"compute_environment_order": [
{
"order": 1,
"compute_environment": "demo-batch-compute-env"
}
]
}
}
Parameters:
-
resource_type (string) [REQUIRED] Resource type.
-
job_queue_name (string) [REQUIRED] The name of the job queue. Up to 128 letters (uppercase and lowercase), numbers, and underscores are allowed.
-
state (string) Should be one of available values: "ENABLED", "DISABLED"; The default value is
ENABLED -
priority (int) [REQUIRED] The priority of the job queue. Job queues with a higher priority (or a higher integer value for the priority parameter) are evaluated first when associated with the same compute environment. Priority is determined in descending order.
-
compute_environment_order (list) The set of compute environments mapped to a job queue and their order relative to each other. Should be described as a list of dicts with the following format:
[
{
"order": 123,
"compute_environment": "string"
},
{
"order": 124,
"compute_environment": "string"
}
]
- order (int) [REQUIRED] The order of the compute environment. Compute environments are tried in ascending order;
- compute_environment (string) [REQUIRED] Name of the compute environment;
Example job queue:
{
"bestJobQueueEver": {
"resource_type": "batch_jobqueue",
"state": "ENABLED",
"priority": 100,
"compute_environment_order": [
{
"order": 110,
"compute_environment": "some_best_compenv"
},
{
"order": 150,
"compute_environment": "some_not_the_best_compenv"
}
]
}
}
Resource type: AWS Batch Job definition
Command to generate meta:
syndicate generate meta batch_jobdef
--resource_name demo-batch-job
--job_definition_type container
--image public.ecr.aws/amazonlinux/amazonlinux:latest
--job_role_arn arn:aws:iam::987601234050:role/ecsTaskExecutionRole
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-batch-job": {
"resource_type": "batch_jobdef",
"job_definition_type": "container",
"container_properties": {
"image": "public.ecr.aws/amazonlinux/amazonlinux:latest",
"vcpus": 1,
"memory": 1024,
"readonly_root_filesystem": true,
"command": [],
"job_role_arn": "arn:aws:iam::987601234050:role/ecsTaskExecutionRole"
},
"node_properties": {},
"retry_strategy": {}
}
}
Linked resources: iam_role: 4.10 IAM Role
Useful links:
Follow this guide to create the execution IAM role.
Parameters:
-
resource_type (string) [REQUIRED] Resource type.
-
job_definition_name (string) [REQUIRED] The name of the job definition to register. It can be up to 128 letters long. It can contain uppercase and lowercase letters, numbers, hyphens (-), and underscores (_)
-
job_definition_type (string) [REQUIRED] Should be one of available values: "container", "multinode";
-
parameters (dict) Default parameter substitution placeholders to set in the job definition. Parameters are specified as a key-value pair mapping. Parameters in a SubmitJob request override any corresponding parameter defaults from the job definition.
Should be described as a dict of the following format:
{ "string": "string", "string": "string" } -
container_properties (dict)
An object with various properties specific to single-node container-based jobs. If the job definition's type parameter is 'container', then you must specify either 'container_properties' or 'node_properties'.Should be described as a dict of the following format:
{ "image": "string", "vcpus": 123, "memory": 123, "command": [ "string", ], "job_role_arn": "string", "execution_role_arn": "string", "volumes": [ { "host": { "source_path": "string" }, "name": "string" }, ], "environment": [ { "name": "string", "value": "string" }, ], "mount_points": [ { "container_path": "string", "read_only": True|False, "source_volume": "string" }, ], "readonly_root_filesystem": True|False, "privileged": True|False, "ulimits": [ { "hard_limit": 123, "name": "string", "soft_limit": 123 }, ], "user": "string", "instance_type": "string", "resource_requirements": [ { "value": "string", "type": "GPU"|"VCPU"|"MEMORY" }, ], "linux_parameters": { "devices": [ { "host_path": "string", "container_path": "string", "permissions": [ "READ"|"WRITE"|"MKNOD", ] }, ], "init_process_enabled": True|False, "shared_memory_size": 123, "tmpfs": [ { "container_path": "string", "size": 123, "mount_options": [ "string", ] }, ], "max_swap": 123, "swappiness": 123 }, "log_configuration": { "log_driver": "json-file"|"syslog"|"journald"|"gelf"|"fluentd"|"awslogs"|"splunk", "options": { "string": "string" }, "secret_options": [ { "name": "string", "value_from": "string" }, ] }, "secrets": [ { "name": "string", "value_from": "string" }, ], "network_configuration": { "assign_publicIp": "ENABLED"|"DISABLED" }, "fargate_platform_configuration": { "platform_version": "string" } }-
image (string) The image used to start a container. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with
repository-url /image :tag.- Images in Amazon ECR repositories use the full registry and repository URI (for example, 012345678910.dkr.ecr..amazonaws.com/ );
- Images in official repositories on Docker Hub use a single name (for example,
ubuntuormongo); - Images in other repositories on Docker Hub are qualified with an organization name
(for example,
amazon/amazon-ecs-agent); - Images in other online repositories are qualified further by a domain name
(for example,
quay.io/assemblyline/ubuntu).
-
vcpus (int) The number of vCPUs reserved for the job. Each vCPU is equivalent to 1,024 CPU shares.
-
memory (int) This parameter indicates the memory hard limit (in MiB) for a container. If your container attempts to exceed the specified number, it is terminated. You must specify at least 4 MiB of memory for a job using this parameter.
-
command (list) The command that's passed to the container. This parameter maps to
Cmdin theCreate a containersection of the Docker Remote API and theCOMMANDparameter todocker run. -
job_role_arn (string)
The Amazon Resource Name (ARN) of the IAM role that the container can assume for AWS permissions. -
execution_role_arn (string) The Amazon Resource Name (ARN) of the execution role that AWS Batch can assume. For jobs that run on Fargate resources, you must provide an execution role.
-
volumes (list) A list of data volumes used in a job.
Should be described as a list of dicts with the following format:
{ "volumes": [ { "host": { "source_path": "string" }, "name": "string" } ] }- host (dict) The contents of the host parameter determine whether your data volume persists on the host container instance and where it is stored.
-
source_path (string)
The path on the host container instance that's presented to the container. - name (string) The name of the volume.
-
environment (list) The environment variables to pass to a container
Should be described as a list of dicts with the following format:
{ "environment": [ { "name": "string", "value": "string" } ] }- name (string) The name of the environment variable.
- value (string) The value of the environment variable.
-
mount_points (list) The mount points for data volumes in your container Should be described as a list of dicts with the following format:
{ "mount_points": [ { "container_path": "string", "read_only": True | False, "source_volume": "string" } ] }- container_path (string) The path on the container where the host volume is mounted.
-
read_only (bool)
If this value istrue, the container has read-only access to the volume. - source_volume (string) The name of the volume to mount.
-
readonly_root_filesystem (bool) When this parameter is true, the container is given read-only access to its root file system.
-
privileged (bool)
When this parameter is true, the container is given elevated permissions on the host container instance (similar to therootuser). -
ulimits (list) A list of
ulimitsto set in the container.Should be described as a list of dicts with the following format:
{ "ulimits": [ { "hard_limit": 123, "name": "string", "soft_limit": 123 } ] }- hard_limit (int) The hard limit for the ulimit type. [REQUIRED]
- name (string) [REQUIRED] The type of the ulimit.
- soft_limit (int) [REQUIRED] The soft limit for the ulimit type.
-
user (string) The user name to use inside the container.
-
instance_type (string) The instance type to use for a multi-node parallel job. All node groups in a multi-node parallel job must use the same instance type.
This parameter isn't applicable to single-node container jobs or for jobs that run on Fargate resources and shouldn't be provided.
-
resource_requirements (list) The type and amount of resources to assign to a container.
Should be described as a list of dicts with the following format:
{ "resource_requirements": [ { "value": "string", "type": "GPU" | "VCPU" | "MEMORY" }, ] }- value (string) [REQUIRED] The quantity of the specified resource to reserve for the container. The values vary based on the type specified.
-
type (string)
[REQUIRED]
The type of resource to assign to a container.
The supported resources include
GPU,MEMORY, andVCPU.
-
linuxParameters (dict) Linux-specific modifications that are applied to the container, such as details for device mappings.
Should be described as a dicts of the following format:
{ "linux_parameters": { "devices": [ { "host_path": "string", "container_path": "string", "permissions": [ "READ" | "WRITE" | "MKNOD" ] } ], "init_process_enabled": True | False, "shared_memory_size": 123, "tmpfs": [ { "container_path": "string", "size": 123, "mount_options": [ "string" ] } ], "max_swap": 123, "swappiness": 123 } }-
devices (list) Any host devices to expose to the container.
Should be described as a list of dicts with the following format:
{ "devices": [ { "host_path": "string", "container_path": "string", "permissions": [ "READ" | "WRITE" | "MKNOD" ] } ] }- host_path (string) [REQUIRED] The path for the device on the host container instance.
-
container_path (string)
The path inside the container used to expose the host device.
By default, the
host_pathvalue is used. -
permissions (list)
The explicit permissions to provide to the container for the device. By default, the container
has permissions for
read,write, andmknodfor the device.
-
init_process_enabled (bool) If true, run an init process inside the container that forwards signals and reaps processes.
-
shared_memory_size (int) The value for the size (in MiB) of the
/dev/shmvolume. -
tmpfs (list) The container path, mount options, and size (in MiB) of the tmpfs mount.
This parameter isn't applicable to jobs running on Fargate resources and shouldn't be provided. Should be described as a list of dicts with the following format:
{ "tmpfs": [ { "container_path": "string", "size": 123, "mount_options": [ "string" ] } ] }- container_path (string) [REQUIRED] The absolute file path in the container where the tmpfs volume is mounted.
- size (int) [REQUIRED] The size (in MiB) of the tmpfs volume.
-
mount_options (list)
The list of tmpfs volume mount options.
defaults,ro,rw,suid,nosuid,dev,nodev,exec,noexec,sync,async,dirsync,remount,mand,nomand,atime,noatime,diratime,nodiratime,bind,rbind,unbindable,runbindable,private,rprivate,shared,rshared,slave,rslave,relatime,norelatime,strictatime,nostrictatime,mode,uid,gid,nr_inodes,nr_blocks,mpol. -
max_swap (int)
The total amount of swap memory (in MiB) a container can use. - swappiness (int) This allows you to tune a container's memory swappiness behavior. A swappiness value of 0 causes swapping not to happen unless absolutely necessary. A swappiness value of 100 causes pages to be swapped very aggressively. Accepted values are whole numbers between 0 and 100 . If the swappiness parameter isn't specified, a default value of 60 is used.
-
-
log_configuration (dict) The log configuration specification for the container.
Should be described as a dict of the following format:
{ "log_configuration": { "log_driver": "json-file" | "syslog" | "journald" | "gelf" | "fluentd" | "awslogs" | "splunk", "options": { "string": "string" }, "secret_options": [ { "name": "string", "value_from": "string" } ] } }-
log_driver (string) [REQUIRED] The log driver to use for the container. The valid values listed for this parameter are log drivers that the Amazon ECS container agent can communicate with by default.
The supported log drivers are
awslogs,fluentd,gelf,json-file,journald,logentries,syslog, andsplunk. -
options (dict) The configuration options to send to the log driver.
-
secret_options (list) The secrets to pass to the log configuration.
Should be described as a list of dicts with the following format:
{ "secret_options": [ { "name": "string", "value_from": "string" } ] }- name (string) [REQUIRED] The name of the secret.
- value_from (string) [REQUIRED] The secret to expose to the container. The supported values are either the full ARN of the AWS Secrets Manager secret or the full ARN of the parameter in the AWS Systems Manager Parameter Store.
-
-
secrets (list) The secrets for the container. Should be described as a list of dicts with the following format:
{ "secrets": [ { "name": "string", "value_from": "string" } ] }- name (string) [REQUIRED] The name of the secret.
- value_from (string) [REQUIRED] The secret to expose to the container. The supported values are either the full ARN of the AWS Secrets Manager secret or the full ARN of the parameter in the AWS Systems Manager Parameter Store.
-
network_configuration (dict) The network configuration for jobs running on Fargate resources. Jobs running on EC2 resources must not specify this parameter.
Should be described as a dict of the following format:
{ "network_configuration": { "assign_public_ip": "ENABLED" | "DISABLED", } }- assign_public_ip (string) Indicates whether the job should have a public IP address.
-
fargate_platform_configuration (dict) The platform configuration for jobs running on Fargate resources. Jobs running on EC2 resources must not specify this parameter. Should be described as a dict of the following format:
{ "fargate_platform_configuration": { "platform_version": "string" } }- platform_version (string) The AWS Fargate platform version where the jobs are running.
-
-
node_properties (dict) An object with various properties specific to multi-node parallel jobs. If you specify node properties for a job, it becomes a multi-node parallel job.
Should be described as a dict of the following format:
{ "num_nodes": 123, "main_node": 123, "node_range_properties": [ { ... }, ] }-
num_nodes (int) [REQUIRED] The number of nodes associated with a multi-node parallel job.
-
main_node (int) [REQUIRED] Specifies the node index for the main node of a multi-node parallel job. This node index value must be fewer than the number of nodes.
-
node_range_properties (list) [REQUIRED] A list of node ranges and their properties associated with a multi-node parallel job.
Should be described as a dict of the following format:
{ "target_nodes": "string", "container": { ... } }-
target_nodes (string) [REQUIRED] The range of nodes, using node index values. A range of 0:3 indicates nodes with index values of 0 through 3 . If the starting range value is omitted (:n ), then 0 is used to start the range. If the ending range value is omitted (n: ), then the highest possible node index is used to end the range. Your accumulative node ranges must account for all nodes (0:n ). You can nest node ranges, for example 0:10 and 4:5 , in which case the 4:5 range properties override the 0:10 properties.
-
container (dict) [REQUIRED] The container details for the node range.
Describes identically to
container_propertiesparameter ofJob Definition
-
-
-
retryStrategy (dict) The retry strategy to use for failed jobs that are submitted with this job definition. Any retry strategy that's specified during a SubmitJob operation overrides the retry strategy defined here. If a job is terminated due to a timeout, it isn't retried.
Should be described as a dict of the following format:
{ "attempts": 123, "evaluateOnExit": [ { "onStatusReason": "string", "onReason": "string", "onExitCode": "string", "action": "RETRY" | "EXIT" } ] }- attempts (int) The number of times to move a job to the RUNNABLE status. You can specify between 1 and 10 attempts.
-
evaluate_on_exit (list)
Array of up to 5 objects that specify conditions under which the job should be retried or failed.
If this parameter is specified, then the attempts parameter must also be specified.
Should be described as a list of dicts with the following keys:
- on_status_reason (string) Contains a glob pattern to match against the StatusReason returned for a job.
- on_reason (string) Contains a glob pattern to match against the Reason returned for a job.
- on_exit_code (string) Contains a glob pattern to match against the decimal representation of the ExitCode returned for a job.
-
action (string)
[REQUIRED]
Specifies the action to take if all of the specified conditions
(
onStatusReason,onReason, andonExitCode) are met.
-
propagate_tags (bool) Specifies whether to propagate the tags from the job or job definition to the corresponding Amazon ECS task. If no value is specified, the tags are not propagated.
-
timeout (dict) The timeout configuration for jobs that are submitted with this job definition, after which AWS Batch terminates your jobs if they have not finished. Should be described as a dict of the following format:
{ "attempt_duration_seconds": 123 }- attempt_duration_seconds (int) The time duration in seconds (measured from the job attempt's startedAt timestamp) after which AWS Batch terminates your jobs if they have not finished. The minimum value for the timeout is 60 seconds.
-
tags (dict) The tags that you apply to the job definition to help you categorize and organize your resources.
-
platform_capabilities (list)
The platform capabilities required by the job definition. If no value is specified, it defaults toEC2. To run the job on Fargate resources, specifyFARGATE. Should be described as list of strings.
Example job definition:
{
"job_definition_to_echo_things": {
"resource_type": "batch_jobdef",
"job_definition_type": "container",
"container_properties": {
"image": "ubuntu",
"vcpus": 1,
"memory": 128,
"command": [
"echo 1"
],
"readonly_root_filesystem": false
}
}
}
Resource type: DocumentDB Cluster
Command to generate meta:
syndicate generate meta documentdb_cluster
--resource_name demo-documentdb-cluster
--master_username demo
--master_password password
--port 27017
--vpc_security_group_ids sg-0aea18793dd1fa3d9
--availability_zones eu-central-1a
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-documentdb-cluster": {
"resource_type": "documentdb_cluster",
"vpc_security_group_ids": [
"sg-0aea18793dd1fa3d9"
],
"availability_zones": [
"eu-central-1a"
],
"port": 27017,
"master_username": "demo",
"master_password": "password"
}
}
Parameters:
"cluster_name": {
"availability_zones": [
"string"
],
"vpc_security_group_ids": [
"string"
],
"port": int,
"master_password": "string",
"master_username": "string"
}
- resource_type (string) [REQUIRED] Resource type.
- cluster_name (string) [REQUIRED] Name of cluster.
- availability_zones (list) A list of Availability Zones that instances in the cluster can be created in.
- vpc_security_group_ids (list) A list of VPC security groups to associate with documentDB cluster.
- port (int) The port number on which the instances in the cluster accept connections.
- master_password (string) The name of the master user for the cluster.
- master_username (string) The password for the master database user.
Example:
"Users": {
"availability_zones": [
"us-east-1a",
"eu-central-1a",
"eu-central-1b",
],
"vpc_security_group_ids": [
"sg-51530134"
],
"port": 27017,
"master_password": "SECURE_password34_#",
"master_username": "root"
}
Resource type: DocumentDB Instance
Command to generate meta:
syndicate generate meta documentdb_instance
--resource_name demo-documentdb-instance
--cluster_identifier demo-documentdb-cluster
--instance_class db.t3.medium
--availability_zone eu-central-1a
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-documentdb-instance": {
"resource_type": "documentdb_instance",
"cluster_identifier": "demo-documentdb-cluster",
"instance_class": "db.t3.medium",
"availability_zone": "eu-central-1a"
}
}
Parameters:
"instance_name": {
"cluster_identifier": "string",
"instance_class": "string",
"availability_zone": "string"
}
- resource_type (string) [REQUIRED] Resource type.
- instance_name (string) [REQUIRED] Name of the instance.
- cluster_identifier (string) [REQUIRED] The cluster identifier that the instance will belong to.
- instance_class (string) [REQUIRED] The compute and memory capacity of the instance(for example db.r5.large, db.r6g.4xlarge, db.t4g.medium etc.
- availability_zone (string) The Amazon EC2 Availability Zone that the instance is created in.
Example:
"new_instance": {
"cluster_identifier": "Users",
"instance_class": "db.r4.xlarge",
"availability_zone": "eu-central-1a"
}
Resource type: Firehose
"stream_type": {
"stream_type": "DirectPut"|"KinesisStreamAsSource"
"kinesis_stream_source_configuration": {
"kinesis_stream_arn": "string",
"role_arn": "string"
},
"delivery_stream_encryption_configuration_input": {
"key_arn": "string",
"key_type": "AWS_OWNED_CM"|"CUSTOMER_MANAGED_CMK"
},
"s3_destination_configuration": {
"role_arn": "string",
"bucket_arn": "string",
"prefix": "string",
"error_output_prefix": "string",
"buffering_hints': {
"size_in_mbs": 123,
"interval_in_seconds": 123
},
"compression_format": "UNCOMPRESSED"|"GZIP"|"ZIP"|"Snappy"|"HADOOP_SNAPPY"
}
}
}
Parameters:
- resource_type (string) [REQUIRED] Resource type.
- stream_name (string) [REQUIRED] Delivery stream name.
- stream_type (string) [REQUIRED] Delivery stream type.
- kinesis_stream_source_configuration (object) If the source for the delivery stream is a Kinesis data stream, this parameter msut contain the Kinesis data stream ARN and the role ARN for the source stream.
- delivery_stream_encryption_configuration_input (object) Used to specify the type and Amazon Resource Name (ARN) of the KMS key needed for Server-Side Encryption (SSE).
- s3_destination_configuration (object) The destination in Amazon S3.
Currently only "AWS" and "AWS_PROXY" lambda integration types are supported
Resource type: Web socket API Gateway
Command to generate meta:
syndicate generate meta web_socket_api_gateway
--resource_name demo-api
--deploy_stage api
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-api": {
"resource_type": "web_socket_api_gateway",
"deploy_stage": "api",
"route_selection_expression": "request.body.action",
"resources": {
"$connect": {
"integration_type": "lambda",
"enable_proxy": true,
"lambda_alias": "${lambdas_alias_name}",
"lambda_name": ""
},
"$disconnect": {
"integration_type": "lambda",
"enable_proxy": true,
"lambda_alias": "${lambdas_alias_name}",
"lambda_name": ""
},
"$default": {
"integration_type": "lambda",
"enable_proxy": true,
"lambda_alias": "${lambdas_alias_name}",
"lambda_name": ""
},
"example": {
"integration_type": "lambda",
"enable_proxy": true,
"lambda_alias": "${lambdas_alias_name}",
"lambda_name": ""
}
}
}
}
Linked resources: api_gateway_resource: 4.5.2 API Gateway Resource lambda: 4.1 LAMBDA
Parameters:
"[api name]": {
"resource_type": "web_socket_api_gateway",
"route_selection_expression": "request.body.action",
"deploy_stage": "test",
"resources": {
"$connect": {
"integration_type": "lambda",
"enable_proxy": true,
"lambda_alias": "${lambdas_alias_name}",
"lambda_name": "[lambda name]"
},
"$disconnect": {
"integration_type": "lambda",
"enable_proxy": true,
"lambda_alias": "${lambdas_alias_name}",
"lambda_name": "[lambda name]"
},
"$default": {
"integration_type": "lambda",
"enable_proxy": true,
"lambda_alias": "${lambdas_alias_name}",
"lambda_name": "[lambda name]"
},
"sendMessage": {
"integration_type": "lambda",
"enable_proxy": true,
"lambda_alias": "${lambdas_alias_name}",
"lambda_name": "[lambda name]"
}
}
}
- resource_type (string) [REQUIRED] Resource type.
- route_selection_expression (string) [REQUIRED] Json path to attribute which is used as a key to select the route;
- deploy_stage (string) [REQUIRED] API deploy stage
- resources (object) [REQUIRED] An object where key is a route (including alpha routes: $connect, $disconnect, $default) and a value is an object with integration_type (string), enable_proxy (bool), lambda_alias (str), lambda_name (str), lambda_version (str).
Resource type: EventBridge rule
Command to generate meta:
syndicate generate meta eventbridge_rule
--resource_name demo-eventbridge-rule
--rule_type schedule
--expression "rate(1 minute)"
--region eu-central-1
JSON Meta generated by the CLI command (deployment_resources.json):
{
"demo-eventbridge-rule": {
"resource_type": "eventbridge_rule",
"rule_type": "schedule",
"region": "eu-central-1",
"expression": "rate(1 minute)"
}
}
Resource type: Eventbridge Schedule
"schedule_name": {
"resource_type": "eventbridge_schedule",
"schedule_content": {
"client_token": "string",
"description": "string",
"end_date": "string",
"flexible_time_window": {
"maximum_window_in_minutes": 123,
"mode": "OFF"|"FLEXIBLE"
},
"group_name": "string",
"kms_key_arn": "string",
"schedule_expression": "string",
"schedule_expression_timezone": "string",
"start_date": "string",
"state": "ENABLED"|"DISABLED",
"target": {
'Arn': 'string',
'DeadLetterConfig': {
'Arn': 'string'
},
'EcsParameters': {
'CapacityProviderStrategy': [
{
'base': 123,
'capacityProvider': 'string',
'weight': 123
},
],
'EnableECSManagedTags': True|False,
'EnableExecuteCommand': True|False,
'Group': 'string',
'LaunchType': 'EC2'|'FARGATE'|'EXTERNAL',
'NetworkConfiguration': {
'awsvpcConfiguration': {
'AssignPublicIp': 'ENABLED'|'DISABLED',
'SecurityGroups': [
'string',
],
'Subnets': [
'string',
]
}
},
'PlacementConstraints': [
{
'expression': 'string',
'type': 'distinctInstance'|'memberOf'
},
],
'PlacementStrategy': [
{
'field': 'string',
'type': 'random'|'spread'|'binpack'
},
],
'PlatformVersion': 'string',
'PropagateTags': 'TASK_DEFINITION',
'ReferenceId': 'string',
'Tags': [
{
'string': 'string'
},
],
'TaskCount': 123,
'TaskDefinitionArn': 'string'
},
'EventBridgeParameters': {
'DetailType': 'string',
'Source': 'string'
},
'Input': 'string',
'KinesisParameters': {
'PartitionKey': 'string'
},
'RetryPolicy': {
'MaximumEventAgeInSeconds': 123,
'MaximumRetryAttempts': 123
},
'RoleArn': 'string',
'SageMakerPipelineParameters': {
'PipelineParameterList': [
{
'Name': 'string',
'Value': 'string'
},
]
},
'SqsParameters': {
'MessageGroupId': 'string'
}
}
}
}
Parameters:
-
client_token (string) Unique, case-sensitive identifier you provide to ensure the idempotency of the request. If you do not specify a client token, EventBridge Scheduler uses a randomly generated token for the request to ensure idempotency.
This field is autopopulated if not provided.
-
description (string) The description you specify for the schedule.
-
end_date (datetime) A date in ISO 8601 or UTC, before which the schedule can invoke its target. Depending on the schedule’s recurrence expression, invocations might stop on, or before, the
end_dateyou specify. EventBridge Scheduler ignoresend_datefor one-time schedules. -
flexible_time_window (string) [REQUIRED] A time window during which EventBridge Scheduler invokes the schedule.
- maximum_window_in_minutes (integer) The maximum time window during which a schedule can be invoked.
- mode (string) [REQUIRED] Determines whether the schedule is invoked within a flexible time window.
-
group_name (string) The name of the schedule group to associate with this schedule. If you omit this, the default schedule group is used.
-
kms_key_arn (string) The Amazon Resource Name (ARN) for the customer managed KMS key that EventBridge Scheduler will use to encrypt and decrypt your data.
-
resource_type (string) [REQUIRED] Resource type.
-
schedule_expression (string) The expression that defines when the schedule runs. The following formats are supported.
-
atexpression - at(yyyy-mm-ddThh:mm:ss) -
rateexpression - rate(value unit) -
cronexpression - cron(fields) You can useatexpressions to create one-time schedules that invoke a target once, at the time and in the time zone, that you specify. You can use rate and cron expressions to create recurring schedules. Rate-based schedules are useful when you want to invoke a target at regular intervals, such as every 15 minutes or every five days. Cron-based schedules are useful when you want to invoke a target periodically at a specific time, such as at 8:00 am (UTC+0) every 1st day of the month.A
cronexpression consists of six fields separated by white spaces: (minutes hours day_of_month month day_of_week year).A
rateexpression consists of a value as a positive integer, and a unit with the following options:minute|minutes|hour|hours|day|daysFor more information and examples, see Schedule types on EventBridge Scheduler in the EventBridge Scheduler User Guide.
-
-
schedule_expression_timezone (string) The timezone in which the scheduling expression is evaluated.
-
schedule_name (string) [REQUIRED] Name of the schedule.
-
start_date (datetime) A date in ISO 8601 or UTC, after which the schedule can begin invoking its target. Depending on the schedule’s recurrence expression, invocations might occur on, or after, the
start_dateyou specify. EventBridge Scheduler ignoresstart_datefor one-time schedules. -
state (string) Specifies whether the schedule is enabled or disabled.
-
target (dict) [REQUIRED] The schedule’s target. Please note that this field follows the default AWS naming convention, which is in
PascalCaseFormat, rather than thesnake_case_format usedfor other fields. It's important to be aware that Syndicate does not perform any validation on this field; data is processed exactly as provided.Target
"Arn":subfield can take universal targets - a customizable set of parameters that allows to invoke a wider set of API operation for many AWS services.To configure a universal target for your schedule using Syndicate you need to specify the following information:
- RoleArn – The ARN for the execution role you want to use for the target. The execution role you specify must have the permissions to call the API operation you want your schedule to target.
-
Arn – The complete service ARN, including the API operation you want
to target, in the following
format:
arn:aws:scheduler:::aws-sdk:service: apiAction. For example, for Amazon SQS, the service name you specify isarn:aws:scheduler:::aws-sdk:sqs:sendMessage. For a complete list of task state resources, see Supported services. -
Input – A well-formed JSON you specify with the request parameters
that EventBridge Scheduler sends to the target API. The parameters and
shape of the JSON you set in
Inputare determined by the service API your schedule invokes. To find this information, see the API reference for the service you want to target.
For more information see: Using universal targets.
Example:
"schedule_name": {
"resource_type": "eventbridge_schedule",
"schedule_content": {
"client_token": "string",
"description": "The description you specify for the schedule",
"end_date": "2023-11-21T12:00:00+00:00",
"flexible_time_window": {
"maximum_window_in_minutes": 123,
"mode": "FLEXIBLE"
},
"group_name": "<your_existing_group>",
"kms_key_arn": "arn:aws:kms:eu-central-1:<your_account_id>:key/<your_key_id>",
"schedule_expression": "rate(24 hour)",
"schedule_expression_timezone": "UTC",
"start_date": "2023-11-20T12:00:00+00:00",
"state": "ENABLED",
"target": {
"Arn": "arn:aws:lambda:eu-central-1:<your_account_id>:function:<your_existing_function_name>",
"Input": "{\"action\": \"start\", \"instance_ids\": [\"i-05f8afc694738f138\"]}",
"RoleArn": "arn:aws:iam::<your_account_id>:role/<your_iam_role_name>"
}
}
}
}
Resource type: DAX Cluster
Command to generate meta:
syndicate generate meta dax_cluster
--resource_name DemoDAXCluster
--node_type dax.t2.small
--iam_role_name DAXServiceRoleForDynamoDBAccess
--subnet_group_name demo-dax-cluster-subnet-group
--cluster_endpoint_encryption_type TLS
JSON Meta generated by the CLI command (deployment_resources.json):
{
"DemoDAXCluster": {
"resource_type": "dax_cluster",
"node_type": "dax.t2.small",
"iam_role_name": "DAXServiceRoleForDynamoDBAccess",
"replication_factor": 3,
"security_group_ids": [],
"availability_zones": [],
"cluster_endpoint_encryption_type": "TLS",
"subnet_group_name": "demo-dax-cluster-subnet-group",
"subnet_ids": []
}
}
Linked resources: iam_role: 4.10 IAM Role
Useful links: Follow this guide to create an IAM service role for DAX to access DynamoDB Follow this guide to create a subnet group for your Amazon DynamoDB Accelerator (DAX).
Resource type: Swagger UI
Command to generate meta:
syndicate generate swagger_ui
--name demo_swagger_ui
--path_to_spec export/demo_spec_oas_v3.json
--target_bucket swagger-ui-demo-bucket
JSON Meta generated by the CLI command (swagger_ui_config.json):
{
"name": "demo_swagger_ui",
"resource_type": "swagger_ui",
"path_to_spec": "export/demo_spec_oas_v3.json",
"target_bucket": "swagger-ui-demo-bucket"
}
Parameters:
- path_to_spec (string) [REQUIRED] Path to OpenAPI specification file of the API gateway for which Swagger UI will be deployed (.json)
-
target_bucket (string) [REQUIRED] S3 bucket with static website hosting preconfigured. For configuring static website hosting by aws-syndicate you can generate meta for the S3 bucket using the option
--static_website_hosting Trueof thegenerate meta s3_bucketcommand
Linked resources: s3_bucket: 4.4 S3 Bucket, api_gateway: 4.5.1 API Gateway