live display of current event status for each stack resource
Would you like to be able to watch the progress of your new CloudFormation stack resources like this? (press play)
<asciinema-player cols=“115” rows=“21” autoplay=“1” font-size=“small” theme=“monokai” title=“aws-cloudformation-stack-status (watching stack-create)” author=“Eric Hammond” author-url=“https://twitter.com/esh" src="/asciinema/201611-aws-cloudformation-stack-status-create.rec” poster=“npt:2:20”
That’s what the output of the new aws-cloudformation-stack-status command looks like when I launch a new AWS Git-backed Static Website CloudFormation stack.
It shows me in real time which resources have completed, which are still in progress, and which, if any, have experienced problems.
Background
AWS provides a few ways to look at the status of resources in a CloudFormation stack including the stream of stack events in the Web console and in the aws-cli.
Unfortunately, these displays show multiple events for each resource (e.g., CREATE_IN_PROGRESS, CREATE_COMPLETE) and it’s difficult to match up all of the resource events by hand to figure out which resources are incomplete and still in progress.
Solution
I created a bit of wrapper code that goes around the aws cloudformation describe-stack-events
command. It performs these
operations:
-
Cuts the output down to the few fields that matter: status, resource name, type, event time.
-
Removes all but the ost recent status event for each stack resource.
-
Sorts the output to put the resources with the most recent status changes at the top.
-
Repeatedly runs this command so that you can see the stack progress live and know exactly which resource is taking the longest.
I tossed the simple script up here in case you’d like to try it out:
You can run it to monitor your CloudFormation stack with this command:
aws-cloudformation-stack-status --watch --region $region --stack-name $stack
Interrupt with Ctrl-C to exit.
Note: You will probably need to start your terminal out wider than 80 columns for a clean presentation.
Note: This does use the aws-cli, so installing and configuring that is a prerequisite.
Stack Delete Example
Here’s another example terminal session watching a stack-delete operation, including some skipped deletions (because of a retention policy). It finally ends with a “stack not found error” which is exactly what we hope for after a stack has been deleted successfully. Again, the resources with the most recent state change events are at the top.
<asciinema-player cols=“115” rows=“21” font-size=“small” theme=“monokai” title=“aws-cloudformation-stack-status (watching stack-delete)” author=“Eric Hammond” author-url=“https://twitter.com/esh" src="/asciinema/201611-aws-cloudformation-stack-status-delete.rec” poster=“npt:1:09”
Note: These sample terminal replays cut out almost 40 minutes of waiting for the creation and deletion of the CloudFront distributions. You can see the real timestamps in the rightmost columns.