Amazon ELB was announced in May and is currently in public beta phase. Previously, HAproxy was the way to go if a customer wanted load balancing within EC2 environment. Amazon ELB is an automatic load balancing solution which detects AWS instance health and distributes traffic accordingly, even across availability zones, but not across regions. The costs of using the service are really not that high, $0.025 per hour and $0.008 per GB transferred.

I wanted to test the ELB service with my simple WordPress installation. I have all the service in one Amazon AWS instance, since, well… I don’t have any sponsors to help with the testing. Thus, I had to setup another instance for the test. Of course, I was then running a duplicate database server as well, but for testing purposes this is all fine. I wanted to test the ELB across availability zones, so I started the instance in eu-west-1a while I had the old instance in eu-west-1b.

Tools:

To get started with Amazon Elastic Load Balancing, ELB API tools must be installed. The tools are installed on your own computer, so you must somehow tell Amazon who you are. There are two ways of describing this to EC2. The tools come with desription in the readme.txt of how to use either one of them. I decided to use the one where each command is appended with descriptions of where the credentials are. You can test the ELB service and your credentials by first querying the currently created load balancers with a command:

C:\Program Files\Support Tools>%AWS_ELB_HOME%\bin\elb-describe-lbs –headers –ec2-cert-filepath=E:\backup_amazon_certs\cert.pem –ec2-private-key-file-path=E:\backup_amazon_certs\pk.pem

The command should return “No LoadBalancers found” if you have not created any and if the command successfully completes.

Installing ELB:

First thing to do is to create a load balancer. The command is:

C:\>%AWS_ELB_HOME%\bin\elb-create-lb wpLoadBalancer –headers –listener “lb-port=80,instance port=80,protocol=HTTP” –availability-zones eu-west-1a –region eu-west-1 –ec2-cert-file path=E:\backup_amazon_certs\cert.pem –ec2-private-key-file-path=E:\backup_amazon_certs\pk.pem

In the above command, I create a load balancer with a name wpLoadBalancer and isntruct the balancer to listen port 80 to incoming requests and also connecting to port 80 in my instances which are to be load balanced. I set the availability zone to be eu-west-1a and the region to be eu-west-1 and the rest is to give information to EC2 of who I am. EC2 will respond with a public DNS name, for example wpLoadBalancer-26728261.eu-west-1.elb.amazonaws.com which doesn’t look too nice, but not to worry, the idea is to create a CNAME record using this information. Of course, this is problematic as you can’t put a CNAME record to the root of the domain.

Next step is to create the health check which the load balancer will use in deciding if the instance is available of not. This is really easy if you know what to do! The load balancer will make make a HTTP GET to the web server for a particular file. If the file is available, the status is OK and the instance is added in the pool of servers. The command is:

C:\>%AWS_ELB_HOME%\bin\elb-configure-healthcheck wpLoadBalancer –headers –region eu-west-1 –target “HTTP:80/ping” –interval 30 –timeout 3 –unhealthy-threshold 2 –healthy-threshold 2 –ec2-cert-file-path=E:\backup_amazon_certs\cert.pem –ec2-private-key-file-path=E:\backup_amazon_certs\pk.pem

EC2 will answer with an ack:
HEALTH-CHECK TARGET INTERVAL TIMEOUT HEALTHY-THRESHOLD UNHEALTHY-THRESHOLD
HEALTH-CHECK HTTP:80/ping 30 3 2 2

This means that the ELB will try to get a file called “ping” from the root of the web server every 30 seconds. The timeout for each request is three seconds and if two pings are missing, the server is removed from the pool until two successful pings are received. I had no previous experience with this, so it took a while to figure out how the ping actually works, but actually all you need is an empty file with that name (or what ever you define in the health check). Important thing is to really keep that empty, because that is just unnecessary traffic if the file would have a size. If the file can’t be found, Apache will respond with a 404 code, while the load balancer wants a 200 code.

Instances are added in the load balancer with the following command:

C:\>%AWS_ELB_HOME%\bin\elb-register-instances-with-lb wpLoadBalancer –headers –region eu-west-1 –instances i-55555555,i-44444444 –ec2-cert-file-path=E:\backup_amazon_certs\cert.pem –ec2-private-key-file-path=E:\backup_amazon_certs\pk.pem

And EC2 responds with:

INSTANCE-ID INSTANCE-ID
INSTANCE-ID i-55555555
INSTANCE-ID i-44444444

One thing to notice here is that you can add instances in the load balancer which are not in the availability zone where the load balancer is. I made a mistake here and were wondering why my other instance’s access log was not getting any hits from the load balancer. The command

C:\>%AWS_ELB_HOME%\bin\elb-describe-instance-health wpLoadBalancer –headers –region eu-west-1 –ec2-cert-file-path=E:\backup_amazon_certs\cert.pem –ec2-private-key-file-path=E:\backup_amazon_certs\pk.pem

And gave a response:

INSTANCE-ID INSTANCE-ID STATE
INSTANCE-ID i-55555555 OutOfService
INSTANCE-ID i-44444444 InService

The problem was fixed by extending the load balancer to cover the eu-west-1b availability zone. How cool is that! Just one command and the balancer covers a new zone! The command was:

C:\>%AWS_ELB_HOME%\bin\elb-enable-zones-for-lb wpLoadBalancer –availability-zones eu-west-1b –headers –region eu-west-1 –ec2-cert-file-path=E:\backup_amazon_certs\cert.pem –ec2-private-key-filepath=E:\backup_amazon_certs\pk.pem

And finally, I had both of my instances with InService state. I could now start the actual balancing of requests for two different sites. I did not do any proper load test by generating load on the instances, I just wanted to test the availability by stopping Apache on either one of the intances. I also edited the www.dkaiser.com CNAME record which was previously pointing to dkaiser.com HOST A record (with elastic IP) and now to the public DNS name of the load balancer.

The first tests were unsuccesful. All was fine when the Apache was running on the instance which had the public Elastic IP configured, suggesting the load balancer did not really distribute traffic to the other node when the first one failed. It just did not work, though the instance was in “InService” state. I then started to google a bit and it became apparent, that the ELB uses a kind of fancy round-robin in distributing requests to a zone. More details here. The point is, there should always be a properly functioning instance per availability zone. This means, having one instance per zone doesn’t really cut it. After I realized my mistake, I had to disable one of the zones from the load balancer, terminate the instance, boot a new instance in eu-west-1b attach volumes and add the new instance to the wpLoadBalancer (once again, how cool is that!). I could now shut down either one of the Apache processes and the site would be up. All as expected.

Summary:

Amazon ELB seems like a great way to load balance traffic. It is also quite cheap. One draw back is the fact that you have to make a CNAME record for the load balancer. This makes it impossible to load balance traffic directed to the root of the site. Fixing this issue is on Amazon’s tasks of future improvements.

Pauli Haikonen