Each November, AWS hosts its annual re:Invent conference in Las Vegas. Drawing over 50,000 people and spanning 8 venues, the conference is not only a draw for AWS partners and customers but also for anyone actively working in the cloud industry. The conference captures the momentum and direction of the cloud industry, while also allowing AWS an enormous platform to launch its new services, some effective immediately and some rolling out over the coming year. While AWS has gotten in the habit of introducing new products and features throughout the year these days, the best stuff still comes out in November. And, the announcements rarely involve AWS playing catchup with their competitors. Typically, this event not only sets what their customers will be working on for the next year but also what their competitors will be. In other words, it’s disruptive. And, it matters. This month, we take a high-level look at what happened this year and what it means for next year. 

-Cris

The Cloud is Moving From Innovation to Maturity

Each year, AWS hosts re:Invent, a sprawling conference that is equal parts pageant and training. Over 50,000 people attended, with thousands of educational sessions and four separate keynote speeches announcing their most significant new products and features of the coming year. While AWS continuously announces updates throughout the year, the best announcements are saved for the megaphone that is re:Invent. This year was no exception.

One of the most significant announcements was the growth of AWS—46% year-over-year, leaving them on track for $27B in revenue in 2018. There are fewprecedents for a company so large growing so fast. The growth is particularly interesting because AWS did not grow its market share over the past year, steadily floating around 52% according to most analysts. AWS is the dominant market leader in the hottest sector of technology—one of the hottest sectors of the economy in fact.

AWS worked hard this year to present a balanced view of all major areas of its platform—compute, databases, analytics, machine learning, IoT, and platform management were all well represented. This highlights that these core areas are maturing yet continuing to evolve. The announcements were consistently significant but never earth-shattering. They did manage to sneak in two blockchain announcements, while calmly pointing out that blockchain’s most touted use case—immutable ledgers—is best implemented differently. And so, they deftly announced a blockchain service and a ledger database at the same time, pleasing both blockchain champions and skeptics at once.

Two of the dominant trends were Kubernetes and cloud security, each receiving significant attention from AWS in its keynotes and each well represented by both vendors and training sessions throughout the event. For very different reasons, each is becoming a critical focus area for businesses migrating to the cloud.

AWS dragged feet on its managed Kubernetes service for a lot longer than most people expected. I suspect this is because of the democratizing power of Kubernetes—in the process of creating a rich, mature platform to run containers, it also becomes a portable environment that can run largely the same on prem, in AWS, Azure or Google. Their initial Kubernetes service, EKS, was met with tepid reviews. Many continued to use other services like Kops to create and manage their clusters, ignoring EKS for quite a while. However, the EKS service matured dramatically over the last year and has significant benefits compared to any other managed Kubernetes service. AWS recognized that its customers wanted Kubernetes so that they could more fully embrace the cloud, not so they could more readily jump ship. There were hundreds of EKS talks this year, with people who had no chance of making it in waiting nonetheless because the topic was such a priority to them. 2018 was the year for Kubernetes to establish itself as the de facto choice for running containers at scale, and we expect 2019 to be the year where Kubernetes adoption for production environments takes off exponentially. AWS is well-positioned to receive a huge share of that growth.

AWS has always focused on security, understanding intuitively that customers needed to understand the vast differences between securing applications in the data center compared to in the cloud. Every re:Invent focuses heavily on security announcements and training, and well funded vendors fill the expo halls with their security solutions. The difference this year was not the focus but rather the maturity of the conversation. AWS has continued to mature its Cloud Adoption Framework and last year published a white paper mapping the NIST Cyber Security Framework (CSF) to the AWS platform. These steps have set clear guidance about what is necessary to properly secure a cloud-native application, and both vendors and the open source community have built heavily around it. The hype of vendors offering silver bullets to cloud security is gone, replaced with mature offerings in well-defined lanes.

The emerging theme this year was undoubtedly maturity. The benefits of cloud-native design are considerable, and the risks are easier to manage than ever with a wide range of feature-filled open source projects, AWS services and third-party vendors. By far, the biggest remaining risk to cloud adoption is merely understanding how to navigate the complex landscape, selecting the correct combination of projects, services and vendors to complement each other, speed your adoption process, save cost and improve security. This is exactly why we remain focused on understanding both the cloud and its supporting ecosystem.

From the Blog

AWS announced new bucket-level controls to restrict public access to buckets. This is a big deal, as prior to this there was no way to definitively make a bucket and its contents private. AWS asserts that buckets were always private by default. In reality, buckets suggested private access in the past rather than enforced it. Bucket-level policies could be overridden by the objects within them via object-level ACLs, and that was dangerous… continue reading

What’s New

AWS News: re:Invent Recap

Another re:Invent is in the books, and there were dozens of major announcements—too many to summarize effectively in a single post. This recap focuses on the ones most relevant to developers and engineers, which means IoT, HPC and machine learning don’t get a lot of coverage here. – Cris

Kubernetes 1.13 release annoucement

The latest 1.13 release for Kubernetes has a few useful beta features finally moving to GA (general availability). Kubeadm allows for bootstrapping Kubernetes clusters using best practices. This tool does not worry about provisioning machines or installing add-ons like monitoring or the Kubernetes dashboard. The Container Storage Interface (CSI) allows for better support for third party storage providers. There should be support for a wider variety of storage solutions in the coming months. One big change in this release is that kube-dns is now being replaced with CoreDNS as the default DNS server in Kubernetes. CoreDNS is both simpler and more flexible than kube-dns. The Kubernetes project does not plan to support kube-dns for much longer so all clusters should be moved to CoreDNS soon. – Kevin

Firecracker (GitHub)

 Firecracker is an open source virtualization framework built on top of KVM. Released by AWS at re:Invent, Firecracker is what powers both Lambda and Fargate. In spite of being a new project, it is literally one of the most heavily used VM platforms in the world. In spite of that, it’s surprisingly easy to get up and running, allowing you to spin up VMs in as little as 125ms. Aside from performance, it offers strong isolation between containers, making it suitable for use in a multi-tenant environment. Native Kubernetes support is expected soon. – Cris

Security Tip

By default, your IAM users that have access to your AWS account do not have to rotate their passwords, have complex passwords, or even use multi-factor authentication. When you set up a new account, the first thing you should do is eliminate usage of the root account—it’s only unique function is the ability to delete the account itself. Create an IAM password policy to force password rotation/complexity. We have a CloudFormation template in our GitHub to set it up for you—create a new stack using the template and you’re done. Then, create a new IAM user and enable multi-factor authentication for it. At that point, you can literally change your root account password and stick it in a safe. – Cris