initial version of EncryptRootVolume Document

[awsandrewpark]

Issue #, if available: N/A

Description of changes: a new Automation Document "EncryptRootVolume" has been created and tested (under Documents/Automation/EncryptRootVolume directory).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[alemartini]

Please implement the pending changes from my first revision, plus the additional ones I posted on the latest changes.

[alemartini]

Good progress.
The main concerns are:

• Add an optional parameter for the snapshot creation wait, with a default value that is good for most of the cases (say, 30 minutes). Most of the times the snapshot creation will be faster, and your wait is dynamic so the execution will be significantly faster. But if the root volume is fairly big and it wasn't snapshot'd recently, it could take hours before it is completed. Have the user pick how much time they are willing to wait.

• Since you have a cleanup branch now, you need to specify the onFailure behavior throughout the document. By default, Automation aborts on failure, but you want to cleanup resources if an intermediate step fails. This is the current graph representation of your SSM document:

// Encrypt Root Volume Automation Document
digraph {
    Start [label=Start]
    End [label=End]
    Start -> describeInstance
    describeInstance -> End [label=onFailure color="red"]
    describeInstance -> describeInstanceRootVolume [label=onSuccess]
    describeInstanceRootVolume -> End [label=onFailure color="red"]
    describeInstanceRootVolume -> createSnapshot [label=onSuccess]
    createSnapshot -> End [label=onFailure color="red"]
    createSnapshot -> extractSnapshotId [label=onSuccess]
    extractSnapshotId -> End [label=onFailure color="red"]
    extractSnapshotId -> copyAndEncryptSnapshot [label=onSuccess]
    copyAndEncryptSnapshot -> End [label=onFailure color="red"]
    copyAndEncryptSnapshot -> waitForSnapshot [label=onSuccess]
    waitForSnapshot -> End [label=onFailure color="red"]
    waitForSnapshot -> createVolume [label=onSuccess]
    createVolume -> handleFailureAndQuit [label=onFailure color="red"]
    createVolume -> stopInstance [label=onFailure color="red"]
    stopInstance -> End [label=onFailure color="red"]
    stopInstance -> detachEBSVolume [label=onSuccess]
    detachEBSVolume -> End [label=onFailure color="red"]
    detachEBSVolume -> attachNewEBSVolume [label=onSuccess]
    attachNewEBSVolume -> End [label=onFailure color="red"]
    attachNewEBSVolume -> applyDeleteOnTerminationValue [label=onSuccess]
    applyDeleteOnTerminationValue -> End [label=onFailure color="red"]
    applyDeleteOnTerminationValue -> restoreInstanceInitialState [label=onSuccess]
    restoreInstanceInitialState -> End [label=onSuccess]
    restoreInstanceInitialState -> End [label=onFailure color="red"]
    handleFailureAndQuit -> End [label=onFailure color="red"]
    handleFailureAndQuit -> handleFailureAndQuit2 [label=onSuccess]
    handleFailureAndQuit2 -> End [label=onSuccess]
    handleFailureAndQuit2 -> End [label=onFailure color="red"]
}

You can visualize it here:

https://dreampuf.github.io/GraphvizOnline

[awsandrewpark]

@alemartini How did you generate the digraph and its visual representation?? It's extremely helpful. I'd love to use it myself.

[alemartini]

@alemartini How did you generate the digraph and its visual representation?? It's extremely helpful. I'd love to use it myself.

See PR #24
I have to submit changes to that PR, but the core code is there.

[alemartini]

I cleaned up the document and changed the cleanup order. The resulting workflow can be visualized from the following dot graph:

// Encrypt Root Volume Automation Document
digraph {
    Start [label=Start]
    End [label=End]
    Start -> describeInstance
    describeInstance -> describeInstanceRootVolume [label=onSuccess]
    describeInstance -> End [label=onFailure color="red"]
    describeInstanceRootVolume -> createSnapshot [label=onSuccess]
    describeInstanceRootVolume -> End [label=onFailure color="red"]
    createSnapshot -> extractSnapshotId [label=onSuccess]
    createSnapshot -> End [label=onFailure color="red"]
    extractSnapshotId -> copyAndEncryptSnapshot [label=onSuccess]
    extractSnapshotId -> deleteRootVolumeSnapshot [label=onFailure color="red"]
    copyAndEncryptSnapshot -> waitForSnapshot [label=onSuccess]
    copyAndEncryptSnapshot -> deleteEncryptedRootVolumeSnapshot [label=onFailure color="red"]
    waitForSnapshot -> createVolume [label=onSuccess]
    waitForSnapshot -> deleteEncryptedRootVolumeSnapshot [label=onFailure color="red"]
    createVolume -> stopInstance [label=onSuccess]
    createVolume -> deleteNewEncryptedVolume [label=onFailure color="red"]
    stopInstance -> detachEBSVolume [label=onSuccess]
    stopInstance -> deleteNewEncryptedVolume [label=onFailure color="red"]
    detachEBSVolume -> attachNewEBSVolume [label=onSuccess]
    detachEBSVolume -> attachOriginalVolume [label=onFailure color="red"]
    attachNewEBSVolume -> applyDeleteOnTerminationValue [label=onSuccess]
    attachNewEBSVolume -> detachNewVolume [label=onFailure color="red"]
    applyDeleteOnTerminationValue -> restoreInstanceInitialState [label=onSuccess]
    applyDeleteOnTerminationValue -> detachNewVolume [label=onFailure color="red"]
    restoreInstanceInitialState -> End [label=onSuccess]
    restoreInstanceInitialState -> End [label=onFailure color="red"]
    detachNewVolume -> attachOriginalVolume [label=onSuccess]
    detachNewVolume -> attachOriginalVolume [label=onFailure color="red"]
    attachOriginalVolume -> deleteNewEncryptedVolume [label=onSuccess]
    attachOriginalVolume -> deleteNewEncryptedVolume [label=onFailure color="red"]
    deleteNewEncryptedVolume -> deleteEncryptedRootVolumeSnapshot [label=onSuccess]
    deleteNewEncryptedVolume -> deleteEncryptedRootVolumeSnapshot [label=onFailure color="red"]
    deleteEncryptedRootVolumeSnapshot -> deleteRootVolumeSnapshot [label=onSuccess]
    deleteEncryptedRootVolumeSnapshot -> deleteRootVolumeSnapshot [label=onFailure color="red"]
    deleteRootVolumeSnapshot -> restoreInstanceInitialState [label=onSuccess]
    deleteRootVolumeSnapshot -> restoreInstanceInitialState [label=onFailure color="red"]
}

New JSON content below. Please test it and submit a new commit. The only other improvement I would like to see (optionally) is to move away from AWS-CreateSnapshot and directly use aws:executeAwsApi and aws:waitForAwsResourceProperty to avoid using the cumbersome logic we have now (createSnapshot -> extractSnapshotId) due to the StringList to String conversion issue.

{
  "schemaVersion": "0.3",
  "description": "Encrypt Root Volume Automation Document",
  "assumeRole": "{{automationAssumeRole}}",
  "parameters": {
    "instanceId": {
      "description": "(Required) Instance ID of the ec2 instance whose root volume needs to be encrypted",
      "type": "String"
    },
    "kmsKeyId": {
      "description": "(Required) Customer KMS key to use during the encryption",
      "type": "String"
    },
    "automationAssumeRole": {
      "type": "String",
      "description": "(Optional) The ARN of the role that allows Automation to perform the actions on your behalf.",
      "default": ""
    }
  },
  "mainSteps": [
    {
      "name": "describeInstance",
      "action": "aws:executeAwsApi",
      "timeoutSeconds": 30,
      "onFailure": "Abort",
      "nextStep": "describeInstanceRootVolume",
      "maxAttempts": 1,
      "inputs": {
        "Service": "ec2",
        "Api": "DescribeInstances",
        "InstanceIds": [
          "{{instanceId}}"
        ]
      },
      "outputs": [
        {
          "Name": "availabilityZone",
          "Selector": "$.Reservations[0].Instances[0].Placement.AvailabilityZone",
          "Type": "String"
        },
        {
          "Name": "rootDeviceName",
          "Selector": "$.Reservations[0].Instances[0].RootDeviceName",
          "Type": "String"
        },
        {
          "Name": "instanceState",
          "Selector": "$.Reservations[0].Instances[0].State.Name",
          "Type": "String"
        }
      ]
    },

    {
      "name": "describeInstanceRootVolume",
      "action": "aws:executeAwsApi",
      "onFailure": "Abort",
      "nextStep": "createSnapshot",
      "timeoutSeconds": 60,
      "maxAttempts": 10,
      "isCritical": true,
      "inputs": {
        "Service": "ec2",
        "Api": "DescribeVolumes",
        "Filters": [
          {
            "Name": "attachment.instance-id",
            "Values": [
              "{{instanceId}}"
            ]
          },
          {
            "Name": "attachment.device",
            "Values": [
              "{{describeInstance.rootDeviceName}}"
            ]
          }
        ]
      },
      "outputs": [
        {
          "Name": "rootDeviceVolumeId",
          "Selector": "$.Volumes[0].Attachments[0].VolumeId",
          "Type": "String"
        },
        {
          "Name": "rootDeviceVolumeType",
          "Selector": "$.Volumes[0].VolumeType",
          "Type": "String"
        },
        {
          "Name": "RootDeviceDeleteOnTermination",
          "Selector": "$.Volumes[0].Attachments[0].DeleteOnTermination",
          "Type": "Boolean"
        }
      ]
    },

    {
      "name": "createSnapshot",
      "action": "aws:executeAutomation",
      "timeoutSeconds": 1800,
      "onFailure": "Abort",
      "nextStep": "extractSnapshotId",
      "maxAttempts": 3,
      "inputs": {
        "DocumentName": "AWS-CreateSnapshot",
        "RuntimeParameters": {
          "VolumeId": "{{describeInstanceRootVolume.rootDeviceVolumeId}}"
        }
      }
    },

    {
      "name": "extractSnapshotId",
      "action": "aws:executeAwsApi",
      "timeoutSeconds": 30,
      "onFailure": "step:deleteRootVolumeSnapshot",
      "nextStep": "copyAndEncryptSnapshot",
      "maxAttempts": 1,
      "inputs": {
        "Service": "ec2",
        "Api": "DescribeSnapshots",
        "SnapshotIds": "{{createSnapshot.Output}}"
      },
      "outputs": [
        {
          "Name": "SnapshotId",
          "Selector": "$.Snapshots[0].SnapshotId",
          "Type": "String"
        }
      ]
    },

    {
      "name": "copyAndEncryptSnapshot",
      "action": "aws:executeAwsApi",
      "timeoutSeconds": 3600,
      "onFailure": "step:deleteEncryptedRootVolumeSnapshot",
      "nextStep": "waitForSnapshot",
      "maxAttempts": 1,
      "inputs": {
        "Service": "ec2",
        "Api": "CopySnapshot",
        "SourceSnapshotId": "{{extractSnapshotId.SnapshotId}}",
        "SourceRegion": "{{global:REGION}}",
        "Encrypted": true,
        "KmsKeyId": "{{kmsKeyId}}",
        "DestinationRegion": "{{global:REGION}}"
      },
      "outputs": [
        {
          "Name": "encryptedSnapshotId",
          "Selector": "$.SnapshotId",
          "Type": "String"
        }
      ]
    },

    {
      "name": "waitForSnapshot",
      "action": "aws:waitForAwsResourceProperty",
      "timeoutSeconds": 1800,
      "onFailure": "step:deleteEncryptedRootVolumeSnapshot",
      "nextStep": "createVolume",
      "inputs": {
        "Service": "ec2",
        "Api": "DescribeSnapshots",
        "SnapshotIds": [
          "{{copyAndEncryptSnapshot.encryptedSnapshotId}}"
        ],
        "PropertySelector": "$.Snapshots[0].State",
        "DesiredValues": [
          "completed"
        ]
      }
    },

    {
      "name": "createVolume",
      "action": "aws:executeAwsApi",
      "timeoutSeconds": 30,
      "onFailure": "step:deleteNewEncryptedVolume",
      "nextStep": "stopInstance",
      "maxAttempts": 1,
      "inputs": {
        "Service": "ec2",
        "Api": "CreateVolume",
        "AvailabilityZone": "{{describeInstance.availabilityZone}}",
        "Encrypted": true,
        "KmsKeyId": "{{kmsKeyId}}",
        "SnapshotId": "{{copyAndEncryptSnapshot.encryptedSnapshotId}}",
        "VolumeType": "{{describeInstanceRootVolume.rootDeviceVolumeType}}"
      },
      "outputs": [
        {
          "Name": "NewRootVolumeID",
          "Selector": "$.VolumeId",
          "Type": "String"
        }
      ]
    },

    {
      "name": "stopInstance",
      "action": "aws:executeAutomation",
      "timeoutSeconds": 300,
      "onFailure": "step:deleteNewEncryptedVolume",
      "nextStep": "detachEBSVolume",
      "maxAttempts": 1,
      "inputs": {
        "DocumentName": "AWS-StopEC2Instance",
        "RuntimeParameters": {
          "InstanceId": "{{instanceId}}"
        }
      }
    },

    {
      "name": "detachEBSVolume",
      "action": "aws:executeAutomation",
      "timeoutSeconds": 300,
      "onFailure": "step:attachOriginalVolume",
      "nextStep": "attachNewEBSVolume",
      "maxAttempts": 1,
      "inputs": {
        "DocumentName": "AWS-DetachEBSVolume",
        "RuntimeParameters": {
          "VolumeId": "{{describeInstanceRootVolume.rootDeviceVolumeId}}"
        }
      }
    },

    {
      "name": "attachNewEBSVolume",
      "action": "aws:executeAutomation",
      "timeoutSeconds": 180,
      "onFailure": "step:detachNewVolume",
      "nextStep": "applyDeleteOnTerminationValue",
      "maxAttempts": 1,
      "inputs": {
        "DocumentName": "AWS-AttachEBSVolume",
        "RuntimeParameters": {
          "Device": "{{describeInstance.rootDeviceName}}",
          "InstanceId": "{{instanceId}}",
          "VolumeId": "{{createVolume.NewRootVolumeID}}"
        }
      }
    },

    {
      "name": "applyDeleteOnTerminationValue",
      "action": "aws:executeAwsApi",
      "onFailure": "step:detachNewVolume",
      "nextStep": "restoreInstanceInitialState",
      "timeoutSeconds": 60,
      "maxAttempts": 10,
      "isCritical": true,
      "inputs": {
        "Service": "ec2",
        "Api": "ModifyInstanceAttribute",
        "InstanceId": "{{instanceId}}",
        "BlockDeviceMappings": [
          {
            "DeviceName": "{{describeInstance.rootDeviceName}}",
            "Ebs": {
              "DeleteOnTermination": "{{describeInstanceRootVolume.RootDeviceDeleteOnTermination}}"
            }
          }
        ]
      }
    },

    {
      "name": "restoreInstanceInitialState",
      "action": "aws:changeInstanceState",
      "onFailure": "Abort",
      "isCritical": true,
      "isEnd": true,
      "inputs": {
        "InstanceIds": [
          "{{instanceId}}"
        ],
        "DesiredState": "{{describeInstance.instanceState}}"
      }
    },

    {
      "name": "detachNewVolume",
      "action": "aws:executeAutomation",
      "timeoutSeconds": 300,
      "onFailure": "Continue",
      "nextStep": "attachOriginalVolume",
      "maxAttempts": 1,
      "inputs": {
        "DocumentName": "AWS-DetachEBSVolume",
        "RuntimeParameters": {
          "VolumeId": "{{createVolume.NewRootVolumeID}}"
        }
      }
    },

    {
      "name": "attachOriginalVolume",
      "action": "aws:executeAutomation",
      "timeoutSeconds": 180,
      "onFailure": "Continue",
      "nextStep": "deleteNewEncryptedVolume",
      "maxAttempts": 1,
      "inputs": {
        "DocumentName": "AWS-AttachEBSVolume",
        "RuntimeParameters": {
          "Device": "{{describeInstance.rootDeviceName}}",
          "InstanceId": "{{instanceId}}",
          "VolumeId": "{{describeInstanceRootVolume.rootDeviceVolumeId}}"
        }
      }
    },

    {
      "name": "deleteNewEncryptedVolume",
      "action": "aws:executeAwsApi",
      "timeoutSeconds": 300,
      "onFailure": "Continue",
      "nextStep": "deleteEncryptedRootVolumeSnapshot",
      "maxAttempts": 1,
      "inputs": {
        "Service": "ec2",
        "Api": "DeleteVolume",
        "VolumeId": "{{createVolume.NewRootVolumeID}}"
      }
    },

    {
      "name": "deleteEncryptedRootVolumeSnapshot",
      "action": "aws:executeAwsApi",
      "onFailure": "Continue",
      "nextStep": "deleteRootVolumeSnapshot",
      "timeoutSeconds": 300,
      "maxAttempts": 1,
      "inputs": {
        "Service": "ec2",
        "Api": "DeleteSnapshot",
        "SnapshotId": "{{copyAndEncryptSnapshot.encryptedSnapshotId}}"
      }
    },

    {
      "name": "deleteRootVolumeSnapshot",
      "action": "aws:executeAwsApi",
      "onFailure": "Continue",
      "nextStep": "restoreInstanceInitialState",
      "timeoutSeconds": 300,
      "maxAttempts": 1,
      "inputs": {
        "Service": "ec2",
        "Api": "DeleteSnapshot",
        "SnapshotId": "{{extractSnapshotId.SnapshotId}}"
      }
    }

  ],
  "outputs":[
    "describeInstanceRootVolume.rootDeviceVolumeId",
    "createVolume.NewRootVolumeID"
  ]
}

[awsandrewpark]

@alemartini Thanks very much for the clean up. I was away for the past couple of days due to moving. I should be able to resume the efforts on this. I'll test out the code you sent and will commit it after testing.