With the endless flood of new products, features and changes from AWS and its surrounding ecosystem, it can be easy to miss an update. Our monthly round up highlights major AWS news, announcements, product updates and behind the scenes changes we think are most relevant.

While the marketing team was secretly planning out a virtual replacement for AWS re:Invent 2020, the engineering team was figuring out how and when they were going to announce all of their not-quite-ready-for-GA services. As a result, June was a bit of a quiet month. Still, a few things did happen.

Auto Scaling Groups Now Support Instance Refresh

If you’ve been using ASGs for the past decade or so, you’ve been mystified by the limited support for rolling out a new AMI. Whether using secret functionality in CloudFormation or hacking together Lambdas that kill instances as greatly as possible, there hasn’t been a way to roll out changes that wasn’t unnecessarily cumbersome. But happy days are here (which means more work for you, removing your perfectly fine hacks and replacing them with Instance Refresh).

The key to Instance Refresh is the InstanceWarmup period. This setting gives the instance a chance to fully start up and get ready for production load. While an instance is in this state, it will not be subject to the ASG limits (e.g., max instances) and will not be considered in scale out/scale in decisions. The warmup period does not start until after the instance is marked healthy. As a result, it is important to set this based on the actual time that the instance needs before it’s ready to be counted against scaling metrics, not how long the instance takes from cold start. Not setting this correctly can result in either unwanted scaling events or unnecessarily long rollout times.

It is important to note that the instance refresh only terminates the instances in a graceful fashion, replacing them according to the in-place configuration. It is up to the user to update the relevant launch config or launch template prior to requesting the refresh.

Considerations

• Instance Refresh depends on a useful healthcheck to make decisions on when to advance to the next instance. If you’re using an EC2 healthcheck instead of an ELB healthcheck, you’re probably not going to like how this works.
• If the refresh fails, it will stop. This means any instances that were successfully deployed will stay, along with your old instances. Be sure to monitor the results of the refresh, either through automation or manual verification.
• Related to the above, there is not a secret blue/green deployment strategy hiding within this feature.

Lambda Adds EFS Support

With last month’s announcement that ECS and Fargate now support EFS, hearing that AWS Lambda also supports EFS now might sound exciting. It’s not. Don’t do this. If for no other reason than because your Lambda probably isn’t as secure as you think it is, and the data you’re storing in EFS is probably more sensitive than you realize.

That’s all.

Redshift Expands External Data Support

In two separate announcements, AWS announced support for writing to external tables in S3 and materialized views that reference external data sources. While these are two useful features in and of themselves, they also point toward AWS’ strategy to backstop revenue loss as people migrate to dramatically more cost effective storage backends like Athena (aka Presto).

While S3-based warehouses offer storage savings that is impossible to ignore, they also have drawbacks. In particular, query times can vary wildly, making them unsuitable for real-time functions or user-facing interfaces. Query limitations can also make routine queries require data structuring, meaning you have to invest heavily up front in understanding exactly how you intend to use your data. Redshift, on the other hand, is more prepared to meet you where you’re at–and bill you lots of compute as needed.

By allowing a two-way flow of data between external data sources, AWS presents the best of both worlds and allows customers to gain the cost savings from not having unneeded data in hot, heavily indexed storage, while also still providing RDBMS-like capabilities. These two features being released together is no coincidence. There are all kinds of linked use cases. An overly simplistic example would be to stream data into an Athena table via Redshift, with a materialized view that constrains a rolling view of the data set.

Miscellaneous News

• App Mesh adds native Kubernetes support with open source controller. While App Mesh has always supported EKS, the integration was at arm’s length and not as rich as other service meshes that more natively support Kubernetes. On the other hand, App Mesh also allows you to bring in Lambda, ECS and other compute platforms for your services. With the new controller, App Mesh offers the best of both worlds. Of course, it’s new, and they are committing frequently working out new issues and adding essential features that weren’t in the initial release. It’s worth noting that this is a rare instance of AWS’ marketing department referring to a project in a GitHub repo as “GA”.
• RDS now supports PostgreSQL 13 Beta 1 (in preview). This version of Postgres has a few neat features, but mostly this is good news just because of how quickly after release AWS has brought support for a beta on a major new release. Their support of PostgreSQL as a first class citizen has come a long way.
• C6g and R6g Graviton2 instances available. There’s some reasonable scuttle about how much faster Graviton2 is (or even isn’t) compared to x86. And indeed, the appeal of ARM architecture favors the hypervisor more than the guest. Still, unless your AWS bill is linked to a credit card with a good frequent flyer program, not trying these out is insane.
• ECS now emits ELB lifecycle events. Previously when ECS tasks would register with multiple target groups, lifecycle events would not be emitted. This could often make debugging issues maddening. Now, other things will step in and make debugging issues maddening.