stepfunctions-tasks: emr-create-cluster - add support for managedScalingPolicy

[kshefchek]

Describe the feature

When creating an EMR cluster in a step function it's possible to create a managed scaling policy when authoring amazon states language json. However, there's no way to do this in cdk that I can see. Would it be possible to add managed scaling policy support in cdk?

Use Case

I want to create an EMR cluster with emr managed scaling in a step function with cdk.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.55.1

Environment details (OS name and version, etc.)

macos

[pahud]

When creating an EMR cluster in a step function it's possible to create a managed scaling policy when authoring amazon states language json

Can you share any links to the aws doc about this? I think it makes perfect sense to support that in aws-stepfunctions-tasks but we just need to know how to configure that correctly with the parameters.

If you have any working sample that might be helpful, feel free to share the code snippets here. Thanks.

[kshefchek]

I could not find any official documentation for this in the step function docs, but it is part of the RunJobFlow API documented here: https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html, ie

...
"LogUri": "string",
"ManagedScalingPolicy": { 
      "ComputeLimits": { 
         "MaximumCapacityUnits": number,
         "MaximumCoreCapacityUnits": number,
         "MaximumOnDemandCapacityUnits": number,
         "MinimumCapacityUnits": number,
         "UnitType": "string"
      }
},
"Name": "string",
...

There is an example here - https://medium.com/@amit.kumarbasu/launch-an-emr-managed-scaling-cluster-with-step-functions-bc544babc80f

[valentinlagunes]

+1