Serverless is problematic

The latest near silver bullet technology seems to be serverless or Function As A Service. Many have trumpeted the goodness of not having to worry about servers or even containers for microservices. But is this the entire story?

I think there are some ways to fix some of the problems but that is a set of ideas for another post.

Impedance Mismatch

It has long been recognized that there are impedance mismatches involved in software systems. In general these are characteristics of the tools, techiques and systems we use that distract from what we are actually seeking to accomplish.

First there is the mismatch of the domain and/or business the software is supposed to provide useful capabilities for and the act of rendering what is desired to some kind of software representation. It has been noted that the choice of software language, for instance, constrains the types of solutions and runnable models that will be proposed to address the need.

Then there are impedance mismatches between the software model in some language and a persistence implementation and/or communications between multiple software components including from what we consider clients to servers as well as peers. These can certainly constrain the type of solutions that are arrived at and how flexible or brittle they are.

Serverless decomposition

Serverless invites, or perhaps forces us, to consider a set of desired functionality in terms of blocks of functionality. There are two level of such decomposition. The first is at the level of what functions or bits of functionality we want to have spun up dynamically when we wish to access that bit of functionality. These are the serverless function callables.

The second level decomposition is which functions are logically part of the same functional group or serverless project. Often things like what dependencies are common to a group of serverless functions or what similarity of logical functionality are present enter in here. But other things like project division considerations among developers also enter in. In many serverless frameworks a project may define many serverless functions and perhaps other types of resources they work with or references to such resources.

For the project we need to gather the required modules the implementation modules may depend on and ultimate create some sort of loadable package of all our implemenation modules and required external or preexisting modules. When a serverless function loads it will load this package, think of it as a sort of tarball and the entry point to the serverless function itself.

Exactly how is it reasonable to break a set of top level functional requirements down into a bunch of small tarballs of loadable code on machines that magically spin up on the fly? How is that natural? A functional completeness and correctness set of concerns is made subservient to a fixed deployment methodology almost from the beginning. This is not a good thing.

Brittleness

Brittleness or lack of flexibility is of two basic types. The is a measure of the difficulty in changing a solution as needs change. The second somewhat related aspect is how likely a change is to cause breakage, especially non-obvious breakage in the solution. This brittleness can become apparent in production or maintenance/development.

Runtime Efficiency

The loading of the above mentioned bundle supporting a lambda function occurs not once for functions in a project although they all have the same underlying dependencies, but for each and every one of these functions when it is loaded.

Their are two cases. In the first case this tarball with entry point, if you will, is already loaded and available. In this case, there is some housekeeping to use it that takes some time, say ~50 ms on AWS, but it otherwise runs immediately. If however it is not loaded then a VM or container load like action occurs which hads, sometimes substantially (up to 3s on AWS in some cases but at least 300ms). So the execution time of the function is not fully predictable. However it is predictable it will take at least the warm load time.

Here at least is one obvious implied place not to use serverless. If you have something you want to execute dependably more than 3–20 times a second and you can simply depend on concurrent fanout (multiple running at once each in their own VM or other instance) then it is a bad match.

Next there is the inefficiency of attempting to get the work of the task done only by calling things within the bundle or going over http[s] to some external to the bundle service by lambda function call or service defined API. There is no general capability to do standard networking between processes / machines as the invocation is completely standalone, ephemeral and isolated.

These ephemeral runtimes can take no advantage of state or internal caching or experience from previous invocations unless it is moved to persistent storage or caching service or somehow signaled or queued externally.

This cannot help but be an efficiency hit except for those things which are only relatively low volume, performance not an issues, unpredictable in number and frequency tasks.

Development Efficiency

The developer needs to learn to think in a rather different way about software solutions. This is inherently an impedance mismatch and case of the machinery driving what the solution can be and how the developer can usefully think about the problem.

Much is made of the lower devops costs of serverless. What actually seems to be true is that each developer creating serverless functions and projects have to think in terms of devops and especially become a deployment engineer. The developer needs to think about how the package that is loaded will be constructed, what it must included, ensure it is not “too big” for the running frametwork, consider its load/deploy time when the lambda function is loaded and so on.

The developer cannot count on any caching or state unless the lambda function is wired up to talk to other services and the allowance for it to do so is hand wired into the lambda function definition. The function can effectively only communicate with thing outside its loadable bundle by either going through some persistence system or using an allowed service or going to some previously created http[s] endpoint or of course invoking another lambda function. There is no capability to open sockets, set up pipes and so on when all serverless invocations are ephemeral and isolated. So a great number of tools in the developer’s toolchest are simply not available or available only in a clumsy, time consuming and quite fragile way. Many of these task used to be devops or system design tasks. But they become the responsiblity or concern of every serverless function developer.

Software can be challenging enough when we are thinking in terms of logical rendering of business needs to efficient, maintable, runnable software implementation. It is much much more challenging when that work is forced into a deployment form that ends up wagging the entire dog of development as serverless does.

Maintainability

For a development team to maintain software in the serverless style they need to understand quite a bit about the serverless frameworks they may be using, the target cloud environment and its services and best practices for each service offered, and the serverless execution environment in detail.

Within all that context they need to understand how the actual business needs were rendered into this context. It is arguably a deep enough overhead that keeping track of the business goals and how they map is not at all an easy task. It is always, of course, a challenge to track real business needs and desires to any as built software system. But I argue the burden is heavier and appreciably more alien to many a development team member using serverless.

Serverless systems in practice have many fragile points such as automatic throttling if over some limit on number of simultaneous serverless functions that can be executing. Or if any of the platform services are performing less well for whatever reason at a particular time than hoped for or expected there can be a world of hurt attempting to find what quite external to the organizations software is the real problem and what if anything can be done about it.

It is not unusual for software architecture to be seriously twisted by experience and folklore concerning the limitations and characteristics of this stack of external dependencies just to go serverless.

Which is why I say the devops problem does not lessen. It fragments and gets put onto the working considerations of every back end team developer.