Skip to content

Commit

Permalink
feat(pipes-targets): add CloudWatch Logs (#30665)
Browse files Browse the repository at this point in the history 
Add CloudWatch Logs as a Pipe target.
  • Loading branch information
@msambol
msambol authored Oct 16, 2024
1 parent 1557793 commit 893769e
Show file tree
Hide file tree
Showing 16 changed files with 33,984 additions and 99 deletions.
231 changes: 132 additions & 99 deletions packages/@aws-cdk/aws-pipes-targets-alpha/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,118 +17,161 @@
<!--END STABILITY BANNER-->


EventBridge Pipes Targets let you create a target for a EventBridge Pipe.
EventBridge Pipes Targets let you create a target for an EventBridge Pipe.

For more details see the service documentation:

[Documentation](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-target.html)
For more details see the [service documentation](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-target.html).

## Targets

Pipe targets are the end point of a EventBridge Pipe.
Pipe targets are the end point of an EventBridge Pipe. The following targets are supported:

* `targets.ApiDestinationTarget`: [Send event source to an EventBridge API destination](#amazon-eventbridge-api-destination)
* `targets.CloudWatchLogsTarget`: [Send event source to a CloudWatch Logs log group](#amazon-cloudwatch-logs-log-group)
* `targets.EventBridgeTarget`: [Send event source to an EventBridge event bus](#amazon-eventbridge-event-bus)
* `targets.KinesisTarget`: [Send event source to a Kinesis data stream](#amazon-kinesis-data-stream)
* `targets.LambdaFunction`: [Send event source to a Lambda function](#aws-lambda-function)
* `targets.SfnStateMachine`: [Invoke a Step Functions state machine from an event source](#aws-step-functions-state-machine)
* `targets.SqsTarget`: [Send event source to an SQS queue](#amazon-sqs)

### Amazon EventBridge API Destination

The following targets are supported:
An EventBridge API destination can be used as a target for a pipe.
The API destination will receive the (enriched/filtered) source payload.

1. `targets.SqsTarget`: [Send event source to an SQS queue](#amazon-sqs)
2. `targets.SfnStateMachine`: [Invoke a State Machine from an event source](#aws-step-functions-state-machine)
3. `targets.LambdaFunction`: [Send event source to a Lambda function](#aws-lambda-function)
4. `targets.ApiDestinationTarget`: [Send event source to an EventBridge API destination](#amazon-eventbridge-api-destination)
5. `targets.KinesisTarget`: [Send event source to a Kinesis data stream](#amazon-kinesis-data-stream)
6. `targets.EventBridgeTarget`: [Send event source to an EventBridge event bus](#amazon-eventbridge-event-bus)
```ts
declare const sourceQueue: sqs.Queue;
declare const dest: events.ApiDestination;

const apiTarget = new targets.ApiDestinationTarget(dest);

### Amazon SQS
const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: apiTarget,
});
```

A SQS message queue can be used as a target for a pipe. Messages will be pushed to the queue.
The input to the target API destination can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetQueue: sqs.Queue;
declare const dest: events.ApiDestination;

const pipeTarget = new targets.SqsTarget(targetQueue);
const apiTarget = new targets.ApiDestinationTarget(dest, {
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: pipeTarget
target: apiTarget,
});
```

The target input can be transformed:
### Amazon CloudWatch Logs Log Group

A CloudWatch Logs log group can be used as a target for a pipe.
The log group will receive the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const targetQueue: sqs.Queue;
declare const targetLogGroup: logs.LogGroup;

const pipeTarget = new targets.SqsTarget(targetQueue,
{
inputTransformation: pipes.InputTransformation.fromObject(
{
"SomeKey": pipes.DynamicInput.fromEventPath('$.body')
})
}
);
const logGroupTarget = new targets.CloudWatchLogsTarget(targetLogGroup);

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: pipeTarget
target: logGroupTarget,
});
```

### AWS Step Functions State Machine
The input to the target log group can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetLogGroup: logs.LogGroup;

const logGroupTarget = new targets.CloudWatchLogsTarget(targetLogGroup, {
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});

A State Machine can be used as a target for a pipe. The State Machine will be invoked with the (enriched) source payload.
const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: logGroupTarget,
});
```

### Amazon EventBridge Event Bus

An EventBridge event bus can be used as a target for a pipe.
The event bus will receive the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStateMachine: sfn.IStateMachine;
declare const targetEventBus: events.EventBus;

const pipeTarget = new targets.SfnStateMachine(targetStateMachine,{});
const eventBusTarget = new targets.EventBridgeTarget(targetEventBus);

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: pipeTarget
target: eventBusTarget,
});
```

Specifying the Invocation Type when the target State Machine is invoked:
The input to the target event bus can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStateMachine: sfn.IStateMachine;
declare const targetEventBus: events.EventBus;

const pipeTarget = new targets.SfnStateMachine(targetStateMachine,
{
invocationType: targets.StateMachineInvocationType.FIRE_AND_FORGET,
}
);
const eventBusTarget = new targets.EventBridgeTarget(targetEventBus, {
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: pipeTarget
target: eventBusTarget,
});
```

The input to the target State Machine can be transformed:
### Amazon Kinesis Data Stream

A Kinesis data stream can be used as a target for a pipe.
The data stream will receive the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStateMachine: sfn.IStateMachine;
declare const targetStream: kinesis.Stream;

const pipeTarget = new targets.SfnStateMachine(targetStateMachine,
{
inputTransformation: pipes.InputTransformation.fromObject({ body: '<$.body>' }),
invocationType: targets.StateMachineInvocationType.FIRE_AND_FORGET,
}
);
const streamTarget = new targets.KinesisTarget(targetStream, {
partitionKey: 'pk',
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: pipeTarget
target: streamTarget,
});
```

The input to the target data stream can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStream: kinesis.Stream;

const streamTarget = new targets.KinesisTarget(targetStream, {
partitionKey: 'pk',
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: streamTarget,
});
```

### AWS Lambda Function

A Lambda Function can be used as a target for a pipe. The Lambda Function will be invoked with the (enriched) source payload.
A Lambda function can be used as a target for a pipe.
The Lambda function will be invoked with the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
Expand All @@ -142,7 +185,7 @@ const pipe = new pipes.Pipe(this, 'Pipe', {
});
```

The target Lambda Function is invoked synchronously by default. You can also choose to invoke the Lambda Function asynchronously by setting `invocationType` property to `FIRE_AND_FORGET`.
The target Lambda function is invoked synchronously by default. You can also choose to invoke the Lambda Function asynchronously by setting `invocationType` property to `FIRE_AND_FORGET`.

```ts
declare const sourceQueue: sqs.Queue;
Expand Down Expand Up @@ -174,102 +217,92 @@ const pipe = new pipes.Pipe(this, 'Pipe', {
});
```

### Amazon EventBridge API Destination
### AWS Step Functions State Machine

An EventBridge API destination can be used as a target for a pipe.
The API destination will receive the (enriched/filtered) source payload.
A Step Functions state machine can be used as a target for a pipe.
The state machine will be invoked with the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const dest: events.ApiDestination;
declare const targetStateMachine: sfn.IStateMachine;

const apiTarget = new targets.ApiDestinationTarget(dest);
const pipeTarget = new targets.SfnStateMachine(targetStateMachine,{});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: apiTarget,
target: pipeTarget
});
```

The input to the target API destination can be transformed:
You can specify the invocation type when the target state machine is invoked:

```ts
declare const sourceQueue: sqs.Queue;
declare const dest: events.ApiDestination;
declare const targetStateMachine: sfn.IStateMachine;

const apiTarget = new targets.ApiDestinationTarget(dest, {
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
const pipeTarget = new targets.SfnStateMachine(targetStateMachine, {
invocationType: targets.StateMachineInvocationType.FIRE_AND_FORGET,
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: apiTarget,
});
```

### Amazon Kinesis Data Stream

A data stream can be used as a target for a pipe. The data stream will receive the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStream: kinesis.Stream;

const streamTarget = new targets.KinesisTarget(targetStream, {
partitionKey: 'pk',
});

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: streamTarget,
target: pipeTarget
});
```

The input to the target data stream can be transformed:
The input to the target state machine can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetStream: kinesis.Stream;
declare const targetStateMachine: sfn.IStateMachine;

const streamTarget = new targets.KinesisTarget(targetStream, {
partitionKey: 'pk',
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});
const pipeTarget = new targets.SfnStateMachine(targetStateMachine,
{
inputTransformation: pipes.InputTransformation.fromObject({ body: '<$.body>' }),
invocationType: targets.StateMachineInvocationType.FIRE_AND_FORGET,
}
);

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: streamTarget,
target: pipeTarget
});
```

### Amazon EventBridge Event Bus
### Amazon SQS Queue

An event bus can be used as a target for a pipe. The event bus will receive the (enriched/filtered) source payload.
An SQS queue can be used as a target for a pipe.
The queue will receive the (enriched/filtered) source payload.

```ts
declare const sourceQueue: sqs.Queue;
declare const targetEventBus: events.EventBus;
declare const targetQueue: sqs.Queue;

const eventBusTarget = new targets.EventBridgeTarget(targetEventBus);
const pipeTarget = new targets.SqsTarget(targetQueue);

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: eventBusTarget,
target: pipeTarget
});
```

The input to the target event bus can be transformed:
The target input can be transformed:

```ts
declare const sourceQueue: sqs.Queue;
declare const targetEventBus: events.EventBus;
declare const targetQueue: sqs.Queue;

const eventBusTarget = new targets.EventBridgeTarget(targetEventBus, {
inputTransformation: pipes.InputTransformation.fromObject({ body: "👀" }),
});
const pipeTarget = new targets.SqsTarget(targetQueue,
{
inputTransformation: pipes.InputTransformation.fromObject(
{
"SomeKey": pipes.DynamicInput.fromEventPath('$.body')
})
}
);

const pipe = new pipes.Pipe(this, 'Pipe', {
source: new SqsSource(sourceQueue),
target: eventBusTarget,
target: pipeTarget
});
```
Loading

0 comments on commit 893769e

Please sign in to comment.