Serverless: What is it and why is it different

What is Serverless Computing?

You may have seen all the recent hype around serverless in the developer community. So what exactly is it? I mean the code must still run somewhere right so how is it actually serverless?

All it means is that the developer and operations team don't have to oversee, manage, or even care about actual servers. This may sound very similar to cloud computing, but there are a few key differences. And these differences mainly lie in what you don't know versus other models.

Operating system unknown!?!?

One of the easy ways to differentiate serverless from some form of fancy cloud orchestration like Kubernetes is that nobody in your company knows what operating system the server that runs your code is using. You may believe since you are running some .Net code it is Windows, or that because it's Ruby it's on Linux, but in the end, you aren't certain and it's of no consequence to your development.

You are able to write code using a language supported by the serverless provider and as long as you stay within the bounds of the box they provide, you are A-Okay with zero knowledge of what operating system, versions, etc that power that.

And in fact, one of the strengths of serverless is that at any given time, your application may be running on multiple different operating systems. All of this is managed and operated by your provider for you.

How many servers do you need to handle your traffic?

If you can answer this question with any sort of guess of around X servers or we need Y CPUs, then you are not doing serverless development.

The serverless contract implies that the number and strength of whatever computing is needed to power your application will not be of any concern to the developer. It doesn't mean you aren't being charged for it, just that it's not something you or your team will manage or care about. Good providers handle automatically managing your service so that it maintains high availability and responsiveness.

Billing model based on compute, storage, and network. Not servers, CPUs, and hard drives

Seeing as how you are not aware of what actual hardware runs underneath your application, a new way to bill comes along with it. Serverless cloud platform providers bill for metered usage of computation, storage, and network transfer. This replaces other models of billing which charge by CPU, disk drives, and network connections. In a serverless world, they are in control of that piece and you are only billed for exact usage of it for your app.

This puts computing much closer to the model of electricity. Your power company bills you by the KWH which is metered usage of electricity. The power itself is created by coal, nuclear, gas, whatever. But the billing is metered out the same no matter what the source.

Idle at zero

The other major change in serverless is that when your app is unused, it automatically scales to zero. Since you are not billed by CPU, but by using computation, then when you are not using it your bill is zero.

The provider is always ready to meter out the computation as needed, but there is no need for you to be paying for a server to be ready for when it gets used. It's simply a calculation of paying for the second of computation actually consumed when your app is executing.

You can think of it a bit like a grocery store. It's their job to keep the store stocked with milk so that when you as a customer are thirsty, you can pop in, buy the milk to drink, and then leave. You don't pay in advance of needing the milk, and you don't pay if they have milk that goes bad because nobody bought it. That is all the grocery store's business is to make sure they have available inventory while being careful not to overstock and waste spoiled milk. All you know is that they have what you want when you want it, and the rest is simplified away from your concern.

Tradeoffs for serverless

So all of this simplification sounds pretty nice. Why wouldn't a person want all of these things: simpler billing, less operational responsibility, easy scaling. Well as with all things, this does come with some tradeoffs. So let's talk about them.

Replace operating system lock-in with cloud provider lock-in

In other models, you had various tradeoffs and limitations due to the operating systems or servers your code ran on. Now that serverless frameworks offload this responsibility to the provider, there are new constraints you have to live by.

As of now, there is not an agreed-upon set of standards defining constraints and guarantees between service providers. That means that similar to how difficult in the past it would be to move applications from Windows to Linux for example. You will now face that when trying to move your serverless apps from Google's Cloud to Amazon's.

These companies do not yet provide any common framework to allow customers to easily move serverless workloads between them. And frankly, it's not in their best interests to do that right now as they would prefer to lock you into their offerings as much as possible. So you need to be very aware that early serverless offerings have many proprietary sticking points to make it tough for you to move out of them.

Less visibility on performance and costs

The tools for diving into the performance of code are very well established for prior programming models. Things like figuring out how much CPU or RAM a given program is using are all commonplace.

With the serverless model, the optimizations change to how much computation, network, and API calls your code uses. To be fair, these are related to the CPU and RAM of the past. But as they get abstracted away even further it will obstruct these tools from being as useful.

I fully believe that new open-source tools for debugging and performance-optimizing will appear to service this market. But they will require a better understanding of how the serverless architecture is being implemented by the providers. It might mean that the cloud vendors are the only ones capable of giving a deep enough view to make these tools effective. And it's not in their interest to help you use fewer resources since they bill for those resources whether you used them efficiently or not.

Long-running applications are not the sweet spot

To gain all of the flexibility that serverless provides, it generally limits the application developer to time-based limitations on these functions as a service. This means it optimizes for allowing your code to respond to a web request in which it has a maximum of 1 minute to respond.

These fixed time maximums assist the provider in being able to fulfill the serverless promises to you. They expect to be able to move around workload between actual physical CPUs and locations as needed in order to provide development teams with a service that auto-scales & heals from equipment failures. Long-running workloads break this assumption. In fact, this is typically listed as one of the requirements of their offering. Where code must complete within X time or be terminated.

For things like web requests, or mobile application APIs, these limitations are not a big deal. But for other use cases like encoding video, operating real-time game servers, or video conferencing solutions these limitations are not feasible. In many cases, you can operate around these limits by creative use of the serverless resources, but you are usually shoehorning in a solution that will cost you more, and operate much more slowly. The cloud providers will be happy to assist you in doing this since more usage is more dollars for them. So be certain you use serverless for web applications and systems where it is the best fit.