#Development
EC2 reserved instances’ silent exit · The impact on Amazon Elastic Compute Cloud users https://ilo.im/16383s
_____
#Amazon #AWS #EC2 #Cloud #Pricing #Application #Website #Hosting #WebDev #Backend
I had to write an automation to rip through 106 Windows Server 2019 #EC2 instances today across 14 accounts in 14 regions. I had to grow all their EBS volumes and notify the OS to grow to the full disk size. And it ran and succeded the first time with 0 failures.
(a) #AWS Systems Manager is kind of amazing
(b) I should go buy a lottery ticket. Nothing that good ever happens when I do stuff like this.
Amazon has this leadership principle of "Learn and Be Curious" which is all about wanting to know things and enjoying learning new things. I have my own version of this called "Learn and Be Furious." Every once in a while I have to learn how something works, and once I get in there and figure it out, I'm shaking my fist at the screen asking "why did they DO it this way!?"
In #AWS EBS volumes are the virtual hard disks on EC2 instances, and EBS volumes can have snapshots. Snapshots are often used for backup/recovery and lots of other important uses, so there is a way to "lock" a snapshot. This prevents it being deleted accidentally. Yesterday I had to learn how to work with locked snapshots.
Here's what I learned.
The API
How do you lock a snapshot? There's an #EC2 modify-snapshot-attribute
API, but "locked" is not a snapshot attribute. You can't lock it that way. Snapshot attributes
are actually mainly permissions. It allows some folks to see, and thereby launch instances from, the snapshot. This is how, say, the Debian team or the FreeBSD team make an AMI that you can launch in EC2. They make an EC2 instance, make a snapshot of its EBS volume, set its snapshot public, and do some other things that make it available. So attributes
aren't really "attributes" in some general sense: they're permissions.
If you want to lock a snapshot there's a lock-snapshot
API. That's all it's good for: locking snapshots. If you want to unlock one, you guessed it: different API: unlock-snapshot
.
This isn't exactly bad. Generally speaking, AWS APIs are service:verb-noun
. So ec2:lock-snapshot
fits the idiom and the common pattern. But by that logic, you'd expect ec2:share-snapshot
and ec2:unshare-snapshot
instead of ec2:modify-snapshot-attributes
with user: all
.
Why so furious?
I'm writing a janitor job that finds orphaned snapshots and deletes them. But if the snapshot is locked, trying to delete it throws an exception.
There are obviously 2 ways to do this: try it anyway and catch the exception when the snapshot is locked and deal with it. Or, I can figure out which snapshots are locked, and don't try to delete them in the first place.
I'm doing the latter, because I guess I want exceptions to be thrown only on failures. I don't want the janitor to run into something I did on purpose (locking a snapshot), and then figure it out down in the exception handler. I guess this is just what I think is the right way to do it, and maybe I'm wrong.
How do I find locked snapshots?
You'd think that you could call describe-snapshots
, which takes certain Filters
. There's a lot of possible things to filter on. I can get it to filter down to a certain set of snapshots based on a few criteria. Locked state is not one of them. In fact, the status of the lock is not returned in the information you get from describe-snapshots
. If you wanted to know about locked snapshots, you should have called describe-locked-snapshots
, which will return just those.
What about the list of unlocked snapshots?
If I have a list of snapshots (say, a list of orphans that should be deleted), but I want to figure out which ones are not locked, how do I do that?
First I get the list of all snapshots (or in my case, all orphaned snapshots). Then I get the list of all locked snapshots. Then I do the diff to remove locked snapshots from the list of all snapshots.
This feels like what my niece would call wonker bonkers. I dunno. Maybe my expectations are all wrong.
I'm working on an EC2 instance. SOMETHING is periodically modifying the routing table to add 169.254.169.253 (aka, the Route53 resolver endpoint) to point to the wrong place, which, naturally, makes everything on the box fall over.
If I delete the offending route, everything starts working again, but then some 15 minutes later it gets added back. I checked all the obvious candidates (cron, systemd, etc) that I could think of and can't figure out what is adding this route. Anybody know of any way to audit routing table changes so that I can stop this from happening anymore?
(this is incidentally a kubernetes node but I don't feel like that should matter???? Idkwtf though)
Leichte Kost zum Abend:
Jemand macht sich gerade an unserem #wordpress zu schaffen.
Unser fail2ban sperrt IP Adressen innerhalb 10 Sekunden.
Der IP Wechsel einer #EC2 in AWS dauert zwischen 1-2 Minuten, je nach dem wie die Cloud Bock hat.
Gerne kannst du das gesamte #aws AS verbrennen, du kommst hier nicht rein. Auch lustig: Er tritt immer wieder in meinen Elementor Honeypot. Wer das auch möchte, um IPs mit fail2ban abzusammeln siehe Bild. Sind noch mehr Plugins als nur #elementor
Discover how ClickHouse unlocked a 25% performance improvement for end users with their AWS Graviton migration.
Insights into their journey and more: https://bit.ly/41gFLUy
FYI: I have made public new #NetBSD AMIs for #AWS #ec2. These are evbarm and amd64 images for NetBSD/10.1 and now have their boot messages sent to the serial console (although it takes EC2 about 5-8 minutes from instance creation until `aws ec2 get-console-output` shows the messages).
These AMIs should be public and ready for you to launch in us-east-1a:
The EC2 instance I'm working out of this morning is 5 hours ahead.
So my stomach sees 3:22pm and it's like, WTF happened to lunch asshole?!
System Administration
Week 1, AWS Aliases
In this video, we demonstrate the use of shell aliases and functions to save ourselves some typing whenever we run #AWS #EC2 commands.
To ensure we get a full dual-stack IPv4 / #IPv6 environment, we're also following this guide:
https://www.netmeister.org/blog/ec2-ipv6.html
The functions shown in the video make use of the subnet and security group described in the blog post and are available here:
https://github.com/jschauma/cloud-functions/blob/main/awsfuncs
Balancing Cloud Infrastructure And AI Ambitions https://www.byteseu.com/564839/ #agents #AI #AmazonQ #AmazonWebServices #ArtificialIntelligence #aws #Bedrock #EC2 #GenerativeAI #S3 #SageMaker
#AWS has just launched the new EC2 C8g & M8g instances!
Powered by the latest Graviton4 processors, these instances promise up to 30% better performance than Graviton3.
Dive into the details: https://bit.ly/3YkldI2
Cool, it seems like if you had an #EC2 instance you HAD it. No way recover data recovery
https://repost.aws/questions/QUAiCPTunxRaWUlY19sg7E6A/recover-ec2-classic-instance
Solution from #AWS is to start a new VPC
#CaseStudy - Figma migrated its compute platform from #AWSECS to #Kubernetes (EKS) in less than 12 months with minimal customer impact.
The result? This move enhances their developer experience, resiliency, and cost efficiency!
More details on #InfoQ https://bit.ly/3XvzkuY