Cloud platforms compared: Heroku, AWS, Azure, Digital Ocean, Google App Engine

A truly all-encompassing comparison of cloud platforms would number among the great works of the century. This is not that work.

If, however, you have a Node app and you have no clue where to host it, I may be able to guide you in a useful direction.

What am I comparing?

Service type: Each of these services exist somewhere along the continuum from Infrastructure (IaaS) to Platform (PaaS), which essentially comes down to the degree to which sysadmin tasks are abstracted away. Of the solutions I am comparing here, the breakdown goes like so:

Ease of use: How quickly can you get up an running with the service?

Performance and Value: These two are tied together. Take a look at the graphs at the end of the post if you want further more detailed info.

Scalabilility: Will your application scale with traffic and further development?

Use Case: Why would you use this service?

Digital Ocean

I am using Digital Ocean (DO) here as a representative of an entire class of virtual machine providers including Linode and Rackspace.

Service type: Pure IaaS. They give you a bare Linux installation on some virtual hardware. You are otherwise totally on your own.

Ease of use: The actual process of creating an account and booting up a "Droplet", as their instances are called, is the simplest of any of the options compared here.

As mentioned though, beyond this point you are totally on your own. If you are setting up a single instance on a single VM this really isn't so bad. You'll need to get comfortable with the command line, and do some googling to figure out nginx configs. Luckily, DO makes it easy to completely reset your instance if you screw things up.

Performance and Value: This is where DO and its ilk really shine. The only option I found to get more performance for your dollar is a 3 year prepay for AWS Micro instances. Additionally, if your app is small enough you can host your database, or even other apps from the same instance.

Scalabilility: They do offer an API for managing and scaling their VMs, but this is really only on the table if you are (or have access to) an experienced sysadmin.

Use Case: Single instance apps (such as this blog) will live quite happily here. You can host quite a few different apps on a single $5/month instance with no noticeable hit to performance. Great for a portfolio of apps that are only accessed occasionaly.

AWS

Service type: Amazon's EC2 instances are themselves more or less purely IaaS, but the full AWS suite offers a wide range of services to optimize and automate some tasks. They also offer PaaS through their Elastic Beanstalk product, but I wasn't able to get it running because their Node documentation is out of date.

Ease of use: AWS has a steeper learning curve than something like Digital Ocean. In addition to an understanding of Linux and nginx at the VM level, you must also wade through a significant amount of AWS specific lingo and concepts.

Performance and Value: AWS is the king here, especially if you're willing to prepay. Also, as IaaS you get the same benefit as DO in being able to host multiple services on one instance. That said, there are some costs that aren't factored into the comparison below, such as load balancing and storage.

Scalabilility: The options here are overwhelming (both a plus and minus). AWS offers a massive suite of options for pretty much any scaling scenario you could imagine, both in terms of resources and tools. You can do everything from a roll your own solution, to different levels of "Application management services" including OpsWorks and CloudFormation.

Use Case: If you have the sysadmin chops to make use of what AWS has to offer there is really no downside here. If you truly want to avoid any possible dead-ends in your solution, AWS should really be your only option. If you don't want to take my word for it, check out Gartner's 2014 IaaS report:

Heroku

Service type: Pure PaaS.

Ease of use: Dead simple. You can be up and running with an app continuously deployed from Github, for free, within minutes. Beautiful interface pared down to the bare essentials.

Performance and Value: Pretty good compared to the other PaaS solutions, but still far off from IaaS.

Scalabilility: It really does not get easier than this. Adjust a slider (or have it scaled automatically) and Heroku will spin-up/down VMs behind the scene scene and handle all of the load balancing for you.

Use Case: If your app fits their platform, they make a pretty compelling case. Dead simple, no configuration, smooth scaling, easy continuous deployment. And on top of all of this, they come at a pretty decent value. Unless you have very significant hosting costs, the difference in price between Heroku and AWS more than likely going to be eaten up with sysadmin person-hours.

Azure

Service type: Azure also comes in both flavors with their "Website" and "Virtual Machine" options.

Ease of use: Azure is in a weird transition at the moment. They have a new admin portal in the works, and my initial impressions have been good, but currently it only offers a subset of the functionality of the old portal, leading me to switch back for the time being.

Their VM interface is pretty smooth, but offers very few configuration options.

Their PaaS offering is really poor compared to the best in class (Heroku).

Performance and Value: The value of their PaaS offering is really dismal and their VMs trail behind both AWS and GCE.

Scalabilility: The auto scaling of the PaaS option is pretty decent, and you can change the instance type of a machine quite smoothly, but Heroku still has them beat.

On the other end of the spectrum their VMs lack the same set of tools that AWS has for scaling large clouds.

Use Case: Less value, more headache. I can't see why you would use Azure, but it's quite possible that there are Windows or .NET needs that make them more attractive.

Google

Service type: Google offers IaaS through Compute Engine and PaaS with App Engine.

Ease of use: Starting up a single VM with Compute Engine was simple, and they offer one of the better in-browser SSH experiences I have encountered. Otherwise they rely pretty heavily on their command line tool, which I found to be clunky and unintuitive.

Performance and Value: Pretty snappy.

Scalabilility: They have the same basic performance scalability as AWS, but the lack of services offered could end up being a liability in the long term.

Use Case: My impression is that Google App Engine requires the most work on the developer's part to get their app ready for deployment. This was a non-starter for this current project, but I would like to check them out further at some point.

Their VMs perform decently, and while my own limited testing didn't bear this out, I've heard that in production they end up cheaper than AWS because they bill in 1 minute increments (minimum 10 per instance) rather than AWS' hour increments.

GCE's load balancing is said to be superior to AWS, handling really sudden traffic spikes better.

And last, though certainly not least, if you need to make use of Google's big data APIs, then GCE is probably a no-brainer.

Performance / Value

For my performance comparison I am using an n-queens solution because it does a good job of maxing out compute performance.

All these options scale in such a way that memory should never be an issue if you're only running Node, which makes it a bit of a moot point.

I would loved to have loved to do a response time comparison of these services, as that would be the other critical element, but that will have to wait for another day.

Note: because this only includes runs for n values up to 14, it favors the lower powered instances. If we were to scale this up to high n values we would see the multi-core VMs pull ahead even more.

This value graph is really the meat of my research. Admittedly this is a very specific use case, with many other elements to factor in, but I think we can draw some reasonable conclusions.

Use the smallest instance that works, and scale horizontally.: The value of the smaller instances is better across the board, no matter which service you are using.

Think twice before using PaaS: They really are quite a bit more expensive, especially if you're hosting multiple services.

tl;dr

If you want to host a small site that will not be getting a huge amount of traffic, and are okay with getting your hands a little dirty with command line and nginx, Digital Ocean is an absolute steal at $5/month. Additionally, you have full control over your machine and can host multiple services.

Heroku is a fantastic option if your app fits their platform. They are free for single instance apps, which is hard to beat, but unlike Digital Ocean you'll have to host your database elsewhere. They can also turn out serious performance when necessary.

AWS is the big leagues. They offer the best value by far, especially if you're willing to commit and prepay. They also have an incredibly deep pool of other services to ensure that they can meet any kind of scalability need you're likely to encounter.

Google is still catching up to Amazon overall, but they have some specific advantages that are compelling.

Lastly we have poor Azure, which doesn't have much to offer compared to it's bretheren.