All posts by Jerome Boyd

Setting Up EC2 Command Line Tools on Windows

There are some great GUI tools for working with EC2 services such as ElasticFox and AWS Management Console.

And that’s just the tip of the iceberg.  However sometimes you need to use the command line tools because you want to script a task, or access features that a GUI tool doesn’t provide access to.  For example today I became motivated to finally get comfortable with the EC2 API so I could create a Elastic Load Balancer instance to test the new functionality provided.

I found lots of tutorials and guidance on setting up your Linux machine to run the tools.  Unfortunately Windows is a 2nd class citizen on AWS.  This is true pretty much across the board from command line tools, to Windows instances (just made available last October, still on Win2k3, etc).

So here is the “definitive guide” to setting up your Windows machine to run the EC2 API command line tools:

Install Java

The first requirement is to have Java 5 or later installed.  If you don’t already have Java installed for some reason go to http://www.java.com/en/download/manual.jsp#win.

Decide on AWS Root

Create a folder called AWS somewhere.  I like to make it easy to get to so I created it at d:\aws.  You can really call this folder whatever you want, but it will be where you store your certificates, your services API files, etc.

Retrieve and Store AWS Certificates

Authentication to AWS happens via a certificate and private key.  You’ll need to retrieve these files from AWS.

Go to http://aws-portal.amazon.com/gp/aws/developer/account/index.html?action=access-key and then scroll down to the X.509 area.  You’ll need to create a new certificate.  Once you do they’ll provide you a Private Key File (pk-<random characters>.pem) and a Certificate (cert-<random characters>.pem).

KEEP THESE FILES PRIVATE.  Possession of these two files give you access to your AWS account.

Configure Environment Variables

Now you need to configure your command line environment with a few environment variables.  Create a batch file in d:\aws called awsTools.bat.  Edit this file with the following text:

REM Path should have bin\java.exe under it
set JAVA_HOME=”C:\Program Files (x86)\java\jre6″

REM Path to Primary Key and Certificate retrieved from AWS
set EC2_PRIVATE_KEY=d:\aws\aws-pk.pem
set EC2_CERT=d:\aws\aws-cer.pem

REM Path to EC2 API, subfolders of bin and lib
set EC2_HOME=d:\aws\ec2
set PATH=%PATH%;%EC2_HOME%\bin

REM Path to ELB API, subfolders of bin and lib
set AWS_ELB_HOME=D:\aws\elb
set PATH=%PATH%;%AWS_ELB_HOME%\bin

cls

cmd

On all of the paths be careful about not including a trailing slash.

JAVA_HOME will need to be set to the appropriate path for your machine.  If you’re confused about where exactly JAVA_HOME should point to find java.exe.  It will be a in a folder called bin.  You want to set JAVA_HOME to the parent directory of bin.

For example on my system you would find java.exe at “C:\Program Files (x86)\java\jre6\bin\java.exe” so I set JAVA_HOME to “C:\Program Files (x86)\java\jre6”

EC2_Private_Key and EC2_Cert both are the location of the private key and certificate that you retrieved from the AWS website in the previous step.  I renamed my key and certificate for simplicities sake.  If you have multiple AWS accounts all you need to do is modify these lines to switch between accounts.

EC2_HOME and AWS_ELB_HOME both point to the folders you unzipped the API into.  Both folders should have two subdirectories called bin and lib.  Bin will contain the cmd files of the different commands for that API.  You set the path variable to include these cmd files in your path so that you do not have to be in that directory to run them.

Now you only need to run the batch file to get a command line with the environmental variables set.  You also could permanently set these variables and have them available in any command window if you choose.  If you want to get fancy you could even put in the logic to set the paths based on the current directory of the batch file, and then put the folder on a thumb drive and carry it around.

Test Command

If you run awsTools.bat you should have a command prompt that you can run the EC2 tools from.  A simple command to test is “ec2-describe-regions”:

D:\aws>ec2-describe-regions

REGION  eu-west-1     eu-west-1.ec2.amazonaws.com

REGION  us-east-1     us-east-1.ec2.amazonaws.com

Amazon Elastic Load Balancer Setup

As I previously wrote about, Amazon announced a load balancing solution called Elastic Load Balancer.  While this may prove to be a great addition to AWS currently none of the GUI tools (including the AWS Console provided by Amazon) have built in functionality to create ELB instances.

So I became motivated to finally get comfortable with the EC2 API, allowing me to call EC2 commands from my windows command line.  I wrote a post detailing how to setup your command line environment for the EC2 API here.

Now armed with a load balancing solution and a working windows command line I wanted to delve into ELB and see what it has to offer.

ELB Documentation

Amazon Web Services in general has excellent documentation.  ELB is no exception.  Probably the most important document you can read is the ELB Quick Reference Card.  This one page sheet shows you all the ELB related commands and their argument options.

ELB Architecture

First a quick overview of the architecture of ELB.  Think of an ELB instance as sitting in front of  your EC2 instances.  ELB routes traffic to your instances you register to be included with ELB.  The ELB instance has it’s own IP address and public DNS name.

As we can see from the diagram the load balancer directs traffic to different instances, even across different availability zones.

One thing to keep in mind is that the requests are balanced between different availability zones and then evenly between the instances of that zone.  So if you have 10 instances in us-east-1a and 5 instances in us-east-1b your us-east-1b instances will service twice as much traffic per instance.  For that reason it is suggested that you keep your number of instances in each zone roughly equal.

When you create the ELB instance it will give you the public DNS name for the instance.  That DNS name will remain the same for the life of the instance.  You will want to create a CNAME record in DNS to point your branded URL (www.mysite.com) to the “ugly” DNS name that EC2 provides you.

Creating ELB Instance

To create an ELB instance first ensure that your command line environment is configured to work with the EC2 API and the ELB API.  I suggest you read my previous article, Setting Up EC2 Command Line Tools on Windows, if you have never use an EC2 command line tool before.

The command for creating an ELB instance is elb-create-lb.  The parameters available on this command are:

<default>Name of Load Balancer; I suggest you use the DNS name of your public service you will be exposing through this ELB instance
–availability-zonesComma delimited list of zones to allow registered EC2 instances in
–listener “protocol=value, lb-port=value, instance-port=value”This defines which protocol and port the ELB instance will listen on, and which port on the instances to send the traffic to.You can have as many –listener parameters as you want.  For example you could configure an ELB instance to listen on ports 80 and 443.

First lets create an ELB instance to listen for HTTP traffic:

d:aws>elb-create-lb Test  –availability-zones us-east-1a,us-east-1b  –listener “protocol=http,lb-port=80,instance-port=80”

DNS-NAME  Test-1736333854.us-east-1.elb.amazonaws.com

As you can see it returns the public DNS name associated with this instance.

Here we create an ELB instance to listen for HTTP and HTTPS traffic:

D:aws>elb-create-lb Test –availability-zones us-east-1a,us-east-1b –listener “protocol=http,lb-port=80,instance-port=80″ –listener “protocol=tcp,lb-port=443,instance-port=443″

DNS-NAME Test-851384903.us-east-1.elb.amazonaws.com

Notice on the protocols we specify HTTP for HTTP traffic, but TCP for HTTPS traffic.  HTTP and TCP are the only protocols supported.

Create CNAME Record for ELB Instance

When you create an ELB instance it provides you a public DNS name.  However they are not user friendly and you will want to create a CNAME record in DNS to redirect your friendly URL to your EC2 hosted website.

How you create the CNAME record depends on who is hosting DNS for you.  However here is the output of my test website I configured for this tutorial:

D:aws>nslookup

Default Server: ip-172-16-0-23.ec2.internal

Address: 172.16.0.23

>aws.LoudSteve.com

Server: ip-172-16-0-23.ec2.internal

Address: 172.16.0.23

Name: Test-5660601.us-east-1.elb.amazonaws.com

Address: 174.129.195.68

Aliases: aws.LoudSteve.com

If you delete your ELB instance and recreate it you will get a new public DNS name and will have to update your CNAME record.

Register EC2 Instance with Load Balancer

Now that you have an ELB instance you need to register EC2 instances with the load balancer.  The command to register an EC2 instance with the ELB instance is elb-register-instances-with-lb.  The parameters available on this command are:

<default>Name of Load Balancer instance to register EC2 instances with.
–instancesComma separated list of instance ID’s

First we need to get a list of our instances because we need the instance ID to register them with the ELB instance.  We do this with ec2-describe-instances from the EC2 API:

D:aws>ec2-describe-instances

<Lots of Stuff>

INSTANCE i-ed156e84   ami-da4daab3

<Lots of Stuff>

INSTANCE i-ef156e86   ami-da4daab3

<Lots of Stuff>

I removed quite a bit from the actual output to help with readability.  The part you want to focus on is where it says “INSTANCE i-**********”.  That is the information you need for each instance.

To register your instances you run the command elb-register-instances-with-lb:

D:aws>elb-register-instances-with-lb Test –instances i-ed156e84, i-ef156e86

INSTANCE-ID  i-ed156e84

INSTANCE-ID  i-ef156e86

You pass it the name of your ELB instance (Test in this case) and a comma separated list of the instance ID’s of your EC2 instances you this load balancer to route traffic to.

To de-register an instance you run the command elb-deregister-instances-from-lb:

D:aws>elb-deregister-instances-from-lb Test –instances i-ed156e84, i-ef156e86

No instances currently registered to LoadBalancer

It takes the same parameters as the register command.

HTTP vs HTTPS

There is not any information on the behavior between HTTP and HTTPS connections available yet.  But I can tell you what I have experienced with my limited tests.

When using HTTP (protocol=http) it appears to not have any session stickiness.  I loaded two web servers with a Default.htm file.  Each file specified which web server I was hitting.  When I repeatedly refreshed the page it bounced back and forth between the two servers pretty consistently.

When using HTTPS (protocol=tcp) the session was sticky.  In fact I could never get it to fail over to the other node.  When I pulled up the page on a different computer though it did pull up the other web server so I know that load balancing was working.

This is far from an extensive test.  I expect more detailed tests and hopefully Amazon themselves will provide specifics soon.

Instance Health Checks

A good load balancer needs a way to check that it’s nodes are online and traffic should still be routed to them.  Otherwise if a node failed the load balancer would continue to route traffic to them and would cause partial downtime for your site.

ELB checks a file that you specify on a schedule that you specify to determine instance health.  You configure this with the elb-configure-healthcheck command.  The parameters are:

<default>Name of Load Balancer instance to configure health checks on.
–targetFile to read
–intervalHow often to perform a health check
–timeoutHow long to allow the server to respond
–unhealthy-thresholdHow many consecutive failed checks before marking node as OutOfService
–healthy-thresholdHow many consecutive successful checks before marking node as InService

Here is an example of configuring health checks:

D:aws>elb-configure-healthcheck Test –target “HTTP:80/status.htm” –interval 5 –timeout 3 –unhealthy-threshold 2 –healthy-threshold 2

HEALTH-CHECK  HTTP:80/status  5  3  2  2

In this example we set the file http://<node IP address>:80/status.htm to be retrieved every 5 seconds.  We allow 3 seconds for the web server to respond.  If it fails to respond after 2 attempts we take the node out of service, if it responds successfully 2 times we put it back in service.

If we run the command elb-describe-instance-health before we configure health checks we will get the following output:

D:aws>elb-describe-instance-health Test

INSTANCE-ID i-ed156e84  InService

INSTANCE-ID i-ef156e86   InService

However once we enable the health checks we get the following output:

D:aws>elb-describe-instance-health Test

INSTANCE-ID i-ed156e84  OutOfService

INSTANCE-ID i-ef156e86   OutOfService

If we looked out our web server logs we would see that the load balancer tried to read the file status.htm and failed.  Once we put that file in place the nodes will go back to being InService.  This is important to note when adding this after you are in production.  You want to have your check file in place before you enable the monitoring.

You should also set that file to not be included in the log file, or you will have an entry in your logs every few seconds while the load balancer checks it’s health.  You should also leave the file blank since there is no reason to increase traffic load with irrelevant data.

Destroying ELB Instance

An ELB instance costs $18/month without even being in use.  Not a huge amount of money, but not something you want to be paying for if your not using it.

To delete an ELB instance you run the command elb-delete-lb:

D:aws>elb-delete-lb Test

Warning: Deleting a LoadBalancer can

lead to service disruption to any

customers connected to the LoadBalancer.

Are you sure you want to delete

this LoadBalancer? [Ny] y

OK-Deleting LoadBalancer

You may want to run elb-describe-lbs to confirm that you no longer have unnecessary ELB instances in place.

Remember if you delete an ELB instance you will not get the same DNS name when you recreate it.  So if you delete it you will have to update your CNAME records to reflect the changes.

Windows Server Core Overview

Beginning with Windows Server 2008 Microsoft offered the option to install the operating system without large parts of the graphical user interface (GUI).  This means when you logon to the server all you get is a command line prompt.  There is no Windows Explorer, no start menu and no Internet Explorer among others.  You want to set the IP address?  Use the command line.  Want to reboot?  Use the command line.  Want to . . . ?  Well you get the idea.

Advantages

System Resources: Server core uses less disk space and less memory.  In short there is less running and less installed.  The full installation of Windows Server 2008 R2 is approximately 7.5gb, while in Core it is approximately 3gb.  The disk space savings is not a serious advantage in my mind unless we are looking at a virtual server environment where you would have dozens or hundreds of these machines using a shared resource (the physical machines storage). A default installation of Windows Server 2008 R2 (no 3rd party apps, no roles installed, etc) consumes 385mb of memory after a reboot.  The same setup but a Core install uses 255mb.  That is a 34% decrease in memory usage.  Multiply that by 100 virtual machines in a VMware or Hyper-V farm and that is a serious resource savings.

Security: Since there is simply less stuff installed there is less to patch and less to attack.  The removal of Internet Explorer alone can reduce the number of patches you install significantly.  A component not installed by Core can not be exploited which will provide significant security enhancements.

Raises Required Skill Level of IT Pro: Managing a Server Core system can be significantly harder if you are not comfortable in a command line environment.  Some readers might think this should be in the Dis-advantages section.  However I see it as an opportunity in two ways. First it sets a minimum skill level for any IT Pro working on the system.  I find Server Core is a great way to keep less experienced administrators away from your critical machines. Secondly it forces you to work smarter.  Windows administrators are plagued with never learning how to do something from the command line (and therefore being able to script it, automate, etc) because the GUI tool will get the job done faster than you can learn what the command syntax is.  Once you put yourself in an environment where you do not have a choice you quickly start to grow as a Windows Administrator.  Starting with Windows Server 2008 R2 the .Net framework is available on core and this means that powershell is available.  The best way for a Windows administrator to jump start their career and make significant gains in their productivity is to learn how to use powershell to manage your servers.

Dis-Advantages

Limited Roles Available:The roles available on core are limited.  In Windows Server 2008 R2 Core the roles available are:

  • Active Directory Certificate Services
  • Active Directory Domain Services
  • Active Directory Lightweight Directory Services (aka ADAM)
  • BranchCache Hosted Cache
  • DHCP Server
  • DNS Server
  • File Services
  • Hyper-V
  • Media Services
  • Print Services
  • Web Server (IIS)
    • Note: In Windows Server 2008 the .Net Framework is not available so that means no ASP.net sites.  In R2 the framework is available.

If you are looking to run a role that is not on the list above you will not be able to use core.  Some common examples are Terminal Services, WSUS, Windows Deployment Services and any other not on the list above.

.Net Framework: As noted in the above section the .Net Framework is not available on Core until R2.  The most notable places this is an issue is if you want to run ASP.net sites or if you want to use PowerShell.  Both of these are pretty painful because core makes a great OS for a web server farm and PowerShell makes a great command line interface to manage an OS from. However unless you have licensing issues preventing you from upgrading there is no reason not to move to R2.  Think of R2 as a really good service pack.  It is an incremental upgrade so it’s already heavily tested, it has a good track record already, and it provides a lot of polish that Windows Server 2008 was missing (such as the .Net Framework on Core). As long as I am encouraging you to upgrade to R2 take note that it is only available in x64 so that means if you have REALLY old hardware you will not be able to run R2.  Also watch out for some 3rd party apps which have not updated their products to officially support R2.  For example VMware (ESX, workstation, etc) has to be upgraded to a certain version to officially support R2 (although I have seen it work on versions that supports Windows Server 2008).

3rd Party Applications: Some third party applications simply do not work without a GUI.  Make sure that your anti-virus, backup agents, monitoring agents, inventory agents, etc will install and allow you to do any management you need to do within core.  One piece of software that is notorious for keeping people off of core is network card management software.  If you want to (for example) team a pair of network cards using the Intel or Broadcom software last I heard those do not work on core.

Conclusion

Windows Server Core in the right situations is a great operating system.  Keep an eye on this blog over the next few weeks as I will be publishing some details on how to manage the core operating system.

Windows Azure Staging Model

One of my favorite features of Windows Azure is their Production/Staging model.  I think the best way to explain why I think this is so well implemented is to walk you through the processes.

Here we have Test1 running in Production and Test2 running in Staging.  Clients that go to your production URL are routed to your production code.  Clients that go to your staging URL are routed to your staging code.  This allows you to have your customers use your production code, while you can test your new code running in staging.

Azure-Stage-Step1

Now that I’ve tested my staging code I am ready to move it to production.  I click the swap button located in-between my two versions and Test2 becomes production and Test1 is moved to staging.  What happens behind the scenes is the load balancer (managed by the Azure platform) starts directing incoming requests to your production URL to servers running the Test2 code base and requests coming into your staging URL are routed to servers running the Test1 code base.  This process literally takes a few seconds since the Test2 code base is already running on servers.  This also gives you the advantage of being able to immediately switch back to your old code base if something goes wrong.

Azure-Stage-Step2

Now we have updated our code again and have pushed up Test3 to our staging area.  We now have Test2 still running in production, and can do testing on Test3 in staging.

Azure-Stage-Step3

Now that we have tested our Test3 code and are ready to move it to production we hit the swap button again and Test3 becomes production and Test2 is moved to staging ready to be moved back into production on a moments notice.

Azure-Stage-Step4

One thing to take note of is that the Web Site URL’s for Production and Staging never changed.  Unfortunately neither of them are URL’s you want your customers to see or you would want work with.  What you want to do is create to DNS CNAME records.  In the example case I’m using here you would create two DNS records:

serktools-stagetest.clodapp.net    CNAME    test.serktools.com

e7e3f38589d04635a6d0d0aee22bd842.cloudapp.net     CNAME     stage.test.serktools.com