(aws-kinesisfirehose-alpha): More than one processor and direct support for the cloudwatch logs decompression

[kbytesys]

Describe the feature

In the AWS panel we have the option to decompress the cloudwatch logs in a really easy way. In the cloud formation this option adds a processor in the "ExtendedS3DestinationConfiguration" below the transform lambda:

    "Type": "Lambda"
 },
 {
    "Type": "Decompression",
    "Parameters": [
        {
            "ParameterName": "CompressionFormat",
            "ParameterValue": "GZIP"
        }
    ]
}

It would be nice to have in cdk:

• a way to use multiple processors for the s3 destination
• configure the cloudwatch log decompression just with a boolean flag

Use Case

It's impossible to configure this feature in cdk without override the Cloudformation node by hand with an high risk of failure and regressions.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.131.0

Environment details (OS name and version, etc.)

Linux Ubuntu 23.10

[pahud]

According to this, extendedS3DestinationConfiguration is actually type CfnDeliveryStream.ExtendedS3DestinationConfigurationProperty and we should have all properties that ExtendedS3DestinationConfigurationProperty has.

a way to use multiple processors for the s3 destination
I am not sure if CFN supports that, do you see any document or sample about it?

[kbytesys]

Yes you got the point, the ProcessingConfiguration property inside ExtendedS3DestinationConfigurationProperty accepts an array of processors instead of just one.

BTW if somebody want to play with the Cfn overrides, this is our solution:

        decompressor = aws_kinesisfirehose.CfnDeliveryStream.ProcessorProperty(
            parameters=[
                aws_kinesisfirehose.CfnDeliveryStream.ProcessorParameterProperty(
                    parameter_name="CompressionFormat", parameter_value="GZIP"
                ),
            ],
            type="Decompression",
        )

        firehose_node = cast(
            aws_kinesisfirehose.CfnDeliveryStream, firehose.node.default_child
        )
        s3_destination = firehose_node.extended_s3_destination_configuration
        s3_destination.processing_configuration.processors.append(decompressor)
        firehose_node.extended_s3_destination_configuration = s3_destination

Pay attention: the last row in this code snippet is not an error, because also if this cdk uses Python, you need to assign the property again because in nodejs that object is immutable and it needs an assignment to understand that the value has been changed.

But again: it would be really nice if we can avoid to play with the Cfn node instead of just have a way to add more than a processor in the aws_kinesisfirehose_destinations_alpha.S3Bucket class.

[pahud]

Thank you for the heads up. I am closing this issue. Feel free to reopen if there's still a concern.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.