How to Avoid Application Failures in the Cloud: Part 2

This is the second in a series of five blog posts that examine how you can build cloud applications that are secure, scalable, and resilient to failures - whether the failures are in the application components or in the underlying cloud infrastructure itself. In the first post, we looked at securing applications. In this post we will look at scalability and availability.

Scalability and Availability

In today’s multi-tiered application architectures, clustering and load-balancing¹ capabilities mean that scalability and availability often go hand-in-hand.

When applications are located on premise, you can configure load-balancing routers to spread connections and inbound traffic across multiple instances of an application in a cluster, providing better response times for users. Load balancing can also provide increased application availability, because the application is less susceptible to the failure of a single application instance. If one does fail, the load balancer can distribute the load over the remaining healthy instances in the cluster. Of course, some sessions or transactions might fail or be rolled back, but the application generally continues to operate unaffected by the instance failure.

Amazon EC2 Elastic Load Balancing (ECB)

Although you don’t have control of the hardware (e.g. routers) used in the Amazon EC2 cloud, you can still implement load balancing strategies for your applications using the Amazon Elastic Load Balancing (ELB) feature². ELB allows you to load balance incoming traffic over a specified number of application instances, with automatic health-checking of each of the application instances. If an instance fails the health check, ELB will stop sending traffic to it.

Figure 2 – Amazon Elastic Load Balancing

Amazon EC2 Auto Scaling

The Amazon EC2 auto scaling feature can dynamically and automatically scale your applications — up or down — based on demand and other conditions (such as response time), so you only have to pay for the compute capacity you actually need and use. This is a case where cloud computing provides a clear cost advantage. If you wanted to be able to dynamically scale your on-premise applications, especially when using virtualization technologies such as VMware or the Xen hypervisor, you would first need to invest in and maintain excess server capacity to handle the peak application demand.

You can define your own Amazon auto-scaling rules to protect your application against slow response times or to ensure that there are enough ”healthy” application instances running to guarantee application availability.

• Availability: You can specify that you always need a minimum of, say, four application instances running to ensure availability to users. The auto-scaling feature will check the health of your application instances to ensure that you have the specified minimum number of instances running. If the number of healthy instances drops below the minimum threshold, the auto-scaling feature will automatically start the required number of instances to restore your application to a healthy state.

• Response time: You can also specify auto-scaling rules based on application response times. For example, you can define a rule to start a new application instance if the response time of the application exceeds 4 seconds for a 15-minute period. If you are using ELB with your application instances, the newly started instances are added to your load balancing group so they can share the user load with the other healthy instances. 

Summary

Given this brief description of load balancing and auto scaling within the Amazon EC2 cloud, you can see how these features can be applied to a multi-tiered application like the one illustrated in Figure 1 to improve scalability and availability. You can imagine that we could use ELB in front of each tier of the application — load balancing across the instances of each security group — and also apply auto-scaling rules to ensure that the application is resilient against an instance failure and can effectively respond to changes in user demand. We will examine a real-life example of combining security groups, load balancing, and auto scaling after we discuss disaster recovery in the next post.

[1] Load balancers provide a host of advanced functionality, including support for sticky user sessions, SSL termination (i.e. handling the SLL processing in the router), and multiple load balancing algorithms.

[2] Amazon ELB capabilities include SSL termination and sticky user sessions, enabling you to implement the same type of load balancing policies as you can with on-premise hardware-based load balancers.