Welcome to the 22nd issue of Last Week in AWS.

Several services in eu-west-1 briefly went down last Wednesday including Lambda, giving me an excuse to finally use this title. Relatedly, I’m “Dublin down” this week– by the time you read this, I’ll be midair on my way to SREcon EMEA. If you’re in Dublin, hit reply; if we meet up, I’ve got Last Week in AWS stickers for you.

I speak Friday afternoon about the inherent hilarity of AWS bills– if you’re at the conference, be sure to come say hello.

Aside from that, last week was suspiciously quiet on the AWS front. “You will regret saying that” followed by raucous laughter was the response I got when I asked AWS employees what was behind that. What’s that ominous rumbling sound?

Community Contributions

Pippa tells us how they built a data pipeline for 400 million podcast listens on AWS, all while remaining… Pippa compliant. I’m so sorry for that pun.

A spirited debate on Reddit around the great question “what the heck do I do with my AWS root account MFA device?” My existing strategy of “lose it” is apparently not recommended.

Cloudonaut graces us with a writeup on how to turn an IOT button into a “turn off the cloud environment” button. It’s nice to see the cloud catch up; physical datacenters have had turn off the datacenter buttons for years– usually pushed accidentally, and usually killing both power to the entire datacenter and someone’s career.

I can’t believe I’ve just discovered this diagram of the various companies and project that comprise the Cloud Native Landscape. Well worth a look, and probably a lot of pull requests. 

Amazon now has as much office space in Seattle as the next 40 largest employers combined. At this rate, they’re about two years away from “you’ll never work in this town again” being literally true.

Last week’s winner of the “unsecured S3 bucket of the week” contest appears to be Groupize. I’m somewhat alarmed that there appears to be no chatter about the blatant PCI-DSS violation inherent to storing CVVs in the first place…

And lastly, Kubernetes advocate / evangelist / podfather Kelsey Hightower fuels speculation about AWS’s next move into container management. While nothing of substance was said, don’t let that stop you from speculating wildly. I sure won’t!

Choice Cuts From the AWS Blog

Amazon Route 53 now supports CAA records - CAA records are DNS records that specify which Certificate Authorities are allowed to issue certificates for a domain. Checking these records becomes mandatory for all CAs on September 8th, so Amazon cut this one a wee bit close! I’m reliving my “wait, was that homework due tomorrow” 10PM panics from my grade-school days.

Monitor your Reserved Instance utilization by receiving alerts via AWS Budgets - You can now get alerted when your RI utilization falls below a definable threshold. Of course, RIs expiring will spike your costs sharply enough that the budget alarms should already alert you– right after an alert would have been really helpful. Of course you missed the “RIs are expiring soon” email; you auto-filter out most AWS email. That was foolish of you.

Analyzing AWS Cost and Usage Reports with Looker and Amazon Athena | AWS Big Data Blog - I love the idea, but I can’t help but shake my head at the complexity behind AWS billing that makes “use our big-data solutions to analyze our bills” not only something the company can post publicly with a straight face, but a legitimately good idea.


If you want to query only instances that have a particular tag or tags, but don’t want to memorize and type in what feels like 40 parameters necessary to get the awscli to do it for you, this script is for you.

We can complain about best practices until we’re blue in the face, but sometimes you need to do something quickly. Holepunch enables that– it quickly allows access to a given security group for an IP (yours by default), and closes the hole again when exited cleanly. You’re going to want to make very sure it exits cleanly.

Tip of the Week

Amazon talks a lot about being “customer obsessed,” and despite the at times excessive level of snark I aim at them, I’ve found it to be generally true. If you go looking for stories of how people killed their companies via AWS overcharges, or gotten themselves into technical binds that AWS couldn’t help with… you generally won’t find them.

Thus, this week’s tip is “give AWS support a chance.” I’m not saying that spending $15K a month minimum for enterprise-tier support makes sense for all use cases, but if you see something terrifying, reach out to Amazon. They’ve been surprisingly friendly / helpful in every case I’ve ever seen. Account / Billing cases are always free.

That said, do understand that they’re generally slammed; one thing they won’t be is “quick to respond”– unless you spend that $15K for enterprise support.

…and that’s what happened Last Week in AWS.

I’m Corey Quinn. I’ve helped people significantly reduce their AWS bills and spoken broadly on the conference circuit, but what I’m good at and passionate about is strategic and tactical decision-making roles at growing startups. If your company is making strides in the tech industry and wants help thinking through these things, get in touch and let’s have a conversation,

If you’ve enjoyed reading this, tell your friends to sign up at lastweekinaws.com (or post a link in your company Slack team!) about it. As always, if you’ve seen a blog post, a tool, or anything else AWS related that you think the rest of the community should hear about, send them my way. You can either hit reply– or join the #lastweekinaws channel on the og-aws Slack team.

List archives are always available at https://snarkive.lastweekinaws.com/