In this blog post, we’ll explore advanced techniques for achieving high availability (HA) on Amazon EC2 on AWS Outposts servers, with a specific focus on stateful applications. This is the second installment of a two-part series, where we will extend the EC2 instance relaunch logic to support workloads that require state persistence across failures.
Importance of High Availability for Stateful Applications on AWS Outposts
Before we get into the specifics of automation, let’s understand why high availability (HA) is crucial for stateful workloads on AWS Outposts servers.
AWS Outposts provide an ideal environment for local computing needs, especially for low-latency, edge processing, and on-premises locations like retail stores, branch offices, healthcare facilities, and more. For stateful applications, you need to preserve instance store data during failovers, making the EC2 auto-relaunch process crucial for maintaining service continuity.
This post builds upon the EC2 auto-relaunch mechanism from Part 1, focusing on how to preserve instance store data using a combination of AWS tools to automate EC2 relaunch with EBS-backed AMIs.
The Architecture Behind High Availability on AWS Outposts Servers
In this architecture, we are focusing on two Outposts servers set up as a redundant pair for fault tolerance. This design allows you to manage EC2 instance relaunch efficiently in the event of a server failure.
For scenarios with three or more servers in the same data center, map each server to a secondary counterpart. This setup ensures high availability. It also ensures that the system seamlessly relaunches EC2 instances during failures.
One of the crucial design components is the use of AWS Systems Manager automation. It creates an EBS-backed AMI in the source region. This ensures that stateful applications retain all necessary data during a server failover event. We convert the EBS volume back to an instance store root volume on the destination Outposts server, ensuring data continuity.
Pro Tip: You can further expand this architecture for multi-region HA to increase redundancy.
Key Prerequisites for Stateful Application Failover on AWS Outposts
Before implementing the solution, ensure the following prerequisites are met:
- EC2 instances to be protected should be Linux-based.
- Instances must have access to Internet Gateway or NAT Gateway for external communication.
- Amazon S3 bucket should be created in the Outposts server’s parent region.
- Set up AWS CloudWatch and configure cross-account observability if necessary.
- Download the repository backup-outposts-servers-linux-instance.
This post builds on the Amazon EC2 auto-relaunch logic implemented in Part 1 of this series. In Part 1, we covered the implementation for achieving HA for stateless applications. In Part 2, we extend the implementation from Part 1 to achieve HA for stateful applications, ensuring they retain EC2 instance store data when we relaunch instances.
Deploying Outposts Servers Linux Instance Backup Solution
For the purpose of this post, we created a virtual private cloud (VPC) named “Production-Application-A” and subnets on each of the two Outposts servers named “source-outpost-a” and “destination-outpost-b.” We supplied the destination-outpost-b subnet in the launch template used for this walkthrough. We have already implemented the Amazon EC2 auto-relaunch logic discussed in Part 1 of this series, and now, we will focus on the next steps required to extend that auto-relaunch capability to stateful applications.
Configuring Systems Manager Automation for Stateful Applications on Outposts Servers
Following the installation instructions available in the GitHub repository README file, you first open an AWS CloudShell terminal from within the account that has access to your Outposts servers. Next, clone the GitHub repository and cd into the “backup-outposts-servers-linux-instance” directory:
From here you can build the Systems Manager Automation document with its attachments using the make documents command. Your output should look similar to the following after successful execution:
Finally, upload the Systems Manager Automation document you just created to the S3 bucket you created in your Outposts server’s parent region for this purpose. For the purpose of this post, an S3 bucket named “ssm-bucket07082024” was created. Following Step 4 in the GitHub installation instructions, the command looks like the following:
BUCKET_NAME="ssm-bucket07082024"
DOC_NAME="BackupOutpostsServerLinuxInstanceToEBS"
OUTPOST_REGION="us-east-1"
aws s3 cp Output/Attachments/attachment.zip s3://${BUCKET_NAME}
aws ssm create-document --content file://Output/BackupOutpostsServerLinuxInstanceToEBS.json --name ${DOC_NAME} -document-type "Automation" --document-format JSON --attachments Key=S3FileUrl,Values=s3://${BUCKET_NAME}/attachment.zip,Name=attachment.zip --region ${OUTPOST_REGION}After creating the Systems Manager Automation document, the command output will show the content of your new file. Review it, then exit the terminal. Confirm that a new file named ‘attachments.zip‘ is in the S3 bucket you specified.
Automating AMI Creation and Launch Template Updates for High Availability on Outposts
Now you’re ready to put this automation logic in place. Following the GitHub instructions for usage, navigate to Systems Manager in the account that has access to your Outposts servers, and execute the automation. We use the default document name, “BackupOutpostsServerLinuxInstanceToEBS,” for the purpose of this post, so we select that document. You may have other documents available for quick setup, but you can disregard those for now.
Select the chosen document to execute this automation using the button in the top right-hand corner of the document details page.
After executing the automation, you are asked to configure the runbook for this automation. Leave the default Simple execution option selected:
For the Input parameters section, review the parameter definitions given in the GitHub repository README file. For the purpose of this post, the following is used:
Note that you may need to create a service role for Systems Manager to perform this automation on your behalf. For the purposes of this post, I have done so by using the Required IAM Permissions to run this runbook section of the GitHub repository README file. The other settings can, however, be left as default. Next, finish your setup by selecting Execute at the bottom of this page. It could take up to 30 minutes for all necessary steps to execute. Keep in mind that the automation document shows 32 steps, but the number of steps executed may vary depending on the type of Linux AMI that you started with. As long as your automation’s overall status shows as successful, you will have completed the implementation successfully. Here is a sample output:
Find the AMI produced from this automation in your Amazon EC2 console under the Images section.
The final implementation
The final implementation step is creating a Systems Manager parameter for the AMI you just created. This prevents you from updating the launch template manually whenever a new AMI is created or its ID changes. The AMI acts as a backup of your application and its instance store state. Expect the AMI ID to change with each new backup or AMI you create. You should also determine the cadence for creating these AMIs to align with your application’s Recovery Point Objectives (RPO).
To create a Systems Manager parameter for your AMI, first navigate to your Systems Manager console. Under Application Management, select Parameter Store and Create parameter. You can select either the Standard or Advanced tier depending on your needs. My AMI ID is ami-038c878d31d9d0bfb, and here is an example of how I fill in the parameter details for this walkthrough:
Updating Launch Template for Automation
Now you can modify your application’s launch template that you created in Part 1 of this series, and specify the Systems Manager parameter you just created. To do this, navigate to your Amazon EC2 console, and under Instances select the Launch Templates option. Create a new version of your launch template, select the Browse more AMIs option, and choose the arrow button to the right of the search bar. Select Specify custom value/Systems Manager parameter.
Now enter the name of your parameter in one of the listed formats, and select Save.
You should see your parameter listed in the launch template summary under Software Image (AMI):
Make sure that your launch template is set to the latest version. Once this is done, your installation is now complete. In the event of a source Outposts server failure, your application will automatically relaunch on a new EC2 instance on your destination Outposts server. Additionally, you will receive a notification email sent to the address specified in the notification email parameter of the init.py script from Part 1 of this series. As a result, you can immediately begin triaging why your source Outposts server experienced a failure, without worrying about getting your application(s) back up and running. This ultimately ensures that your application(s) are highly available and helps reduce your Recovery Time Objective (RTO).
Automating EC2 Instance Relaunch to Achieve Zero-Downtime Failover
The final configuration ensures that in the event of a failure, your EC2 instance will automatically relaunch on the destination Outposts server. This enables high availability and reduced Recovery Time Objective (RTO) for your critical applications.
Important: Make sure you test your HA setup regularly and perform failure drills to ensure your outposts servers are configured properly for disaster recovery.
Conclusion: Empowering High Availability for Stateful Applications with AWS
By leveraging AWS tools like CloudFormation, Lambda, CloudWatch, and Systems Manager, you can build highly available infrastructure for stateful applications running on AWS Outposts servers. This solution automates the relaunch of EC2 instances with preserved instance store data, making it a robust choice for businesses needing resilience and minimal downtime.
Next Steps
- Test your HA setup to ensure it meets your RPO and RTO objectives.
- Monitor your EC2 instance health with Amazon CloudWatch.
- Explore ways to extend your HA architecture with multi-region failovers for added redundancy.
[Image Placeholder: Next Steps Diagram or Flowchart]
Next Steps for Optimizing Your Amazon EC2 on AWS Outposts
If you’re looking to enhance the availability and scalability of your applications, explore Gigabits Cloud Solutions for additional insights into edge computing, AWS Outposts optimization, and high-availability strategies.
Clean-Up and Final Thoughts
The cleanup process is straightforward:
- Delete the CloudFormation stack used for deploying the HA solution.
- Un-enroll EC2 instances from Host Management.
- Delete the EBS-backed AMI to prevent unnecessary costs.
By ensuring a proper cleanup, you avoid resource waste and maintain a clean architecture.
FAQ (Frequently Asked Questions)
Q1: Can this setup be applied to non-Linux instances on Amazon EC2 on AWS Outposts?
A1: We designed this solution specifically for Linux-based EC2 instances running on Amazon EC2 on AWS Outposts.
Q2: How do I manage automatic AMI updates in my launch template for Amazon EC2 on AWS Outposts?
A2: Use the Systems Manager parameter for AMI management to ensure your launch template always references the latest backup AMI on Amazon EC2 on AWS Outposts.
Q3: How do I ensure my EC2 instance relaunch is seamless without data loss on Amazon EC2 on AWS Outposts?
A3: To avoid data loss during EC2 instance relaunch on Amazon EC2 on AWS Outposts, use EBS-backed AMIs to store critical application data. This ensures that your instance store data is preserved even if the instance fails. Make sure to configure Systems Manager automation for AMI creation and relaunch on failover to ensure a seamless transition with zero data loss.
Q4: What are the advantages of using Amazon EC2 on AWS Outposts for stateful applications?
A4: Amazon EC2 on AWS Outposts provides a hybrid cloud solution, allowing you to run AWS infrastructure on-premises. As a result, it offers low-latency computing closer to your data. This is particularly beneficial for stateful applications that require high availability, since it enables automatic failover and continuous data persistence. Additionally, Amazon EC2 on AWS Outposts simplifies networking and security by leveraging the same tools and services available in the AWS cloud.
Q5: How do I test the failover setup in Amazon EC2 on AWS Outposts to ensure it works effectively?
A5: To test your failover setup on Amazon EC2 on AWS Outposts, simulate a failure by manually stopping an EC2 instance on your source Outposts server. Monitor whether the backup EC2 instance on the destination Outposts server launches correctly. Additionally, ensure that all necessary data is intact and the Systems Manager automation is functioning as expected. You can set up CloudWatch Alarms to alert you during failover events and confirm that you meet the RTO (Recovery Time Objective) and RPO (Recovery Point Objective).
Feel free to contact us at Gigabits Cloud for more advanced configurations or custom solutions tailored to your enterprise needs.
