Author: Diego Sucaria

Microservices contract and versioning

What is it? Why is it important?

This is a series of posts I’m doing about designing microservices. Keep posted I will link them up when they are all ready!

When you are designing a microservices architecture, whether you are using a REST or messaging approach for communication between microservices, you have to design the APIs/messages and how a microservice will interact with each other.

One of the most important aspects of the microservices architectures is the ability to work on and deploy a microservice, completely independent of each other. To achieve this, each microservice must provide a well-defined, versioned contract.

The microservices contract

A microservice contract is between the service and its clients. The main goal is that you can make changes to the service without affecting the clients, nevertheless if the clients are aware of the service changes or not.

Even if you spent a lot of time the first time, designing the initial contract, for certain, the API will need to change over time.

When the time comes to update an API, it is important to understand the difference between breaking and non-breaking changes, when a major release is required and when to dispose of an old version.

When the changes are small, for example, adding a new parameter to the API, if that parameter isn’t business-critical, the clients should be able to consume the API in the old way, without sending or expecting to receive that parameter, and the server should fill the blanks with default values.

However, if you are doing a major, backward-incompatible, change to the API, you will need to maintain the old version for some time because you as a service cannot force your clients to update immediately.
If you are using a REST approach, one way is to add a versioning number in the path, for example /app/v1/service, app/v2/service. This way you can have two or more versions of your microservice available.

That is key to understand, if you are not doing a breaking change, there is no need for a new version of your contract.

Running php-fpm and nginx processes in the same container with supervisord

Yes, I know…. containers are not meant to be used like this, running two processes within a single container. Ideally, one container runs only one process.

But, what if we want to have php-fpm + nginx on the same container? since we do need both processes running for serving our website, we may say that there is no benefit in having them in two separate containers, and in case one fails, the other is useless and the whole container should be restarted.

So, we decide to have them both in the same container. How to do it properly?? with supervisord

supervisord

It is a process management daemon that will allow us to monitor and control processes on Linux.
It is quite extensive but we are not going to use all of it. we just need to run two freaking processes.

How to do it?

Dockerfile:

FROM debian:stretch

# make sure you install supervisord
RUN apt-get -qq update > /dev/null && apt-get -qq upgrade -y > /dev/null; \
    apt-get -qq install -y ... supervisor  > /dev/null;

# do your stuff, install php, nginx, whatever do you need.
# .
# .
# after you did everything, set up supervisord

COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf

CMD ["/usr/bin/supervisord"]

The changes in the Dockerfile are straight forward, just:

  • install supervisord
  • copy the config file
  • make docker CMD run supervisord

supervisord.conf:

[supervisord]
nodaemon=true

[program:nginx]
command=nginx -g "daemon off;"
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

[program:php-fpm]
command=/bin/bash -c "mkdir -p /var/run/php && php-fpm7.1 --nodaemonize --fpm-config /etc/php/7.1/fpm/php-fpm.conf"
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

Now things are interesting….. let’s break them up:

  • nodaemon=true we tell supervisord to run as a foreground process.
  • program:nginx we run nginx with the “daemon off” directive, we set it to auto-restart in case it fails and most importantly, we redirect logs to stdout and stderr so that docker can pick them up.
  • program:php-fpm we first create the /var/run/php folder, so php doesn’t fail to start, then we run php-fpm as a foreground process too. We do the same thing we did for nginx, redirecting the logs to stdout and stderr

And…. that is all! you now have php-fpm and nginx running on the same container, in the proper manner, with supervisord supervising them!

Resource management strategy for docker containers on Kubernetes (nodejs + express)

It seems simple, but it is not.

How to properly assign resources requirements and limits to a HTTP dockerized microservice running on kubernetes?

Good question, right?

Well, as you can imagine there is not a single answer to it. But there is a strategy you can follow. This is not a “book definition” what I mean with this is that there might be another way of sizing resource requirements, but, so far, I’ve been using this method successfully.

It is a delicate balance between the hardware size on which your containers will run (ie: k8s nodes) and how many requests the container itself can handle.

The idea is to find that delicate balance so our pods can be scheduled on the nodes, without wasting precious resources that can be used for other workloads.

So, how do we do that?

A simple method:

Let’s take a simple microservice as an example: a nodejs express REST API which talks to a MongoDB.
We will be doing: GET /example

Run the container on your local machine or a k8s cluster, measure the idle resource usage:

  • CPU: 1%
  • Memory: 80Mb


Using a load testing app like Apache JMeter, we fire up 100 concurrent requests in 1 second… measure the resource spike using docker stats or similar:

  • CPU: 140%
  • Memory: 145Mb

We exceeded one CPU core, and that is not good, so we reduce the number of parallel requests to 70

  • CPU: 93%
  • Memory: 135Mb

Now, we have something to work with. We can assume that a single instance of our app, can handle up to 70 requests per second.

Setting Kubernetes resources:

According to how resource limits and requests are set in Kubernetes, we can say that it is safe to consider that our resource requests can be:

  • CPU: 0.1
  • Memory: 100Mb

And we can limit the resources to:

  • CPU: 1
  • Memory: 150Mb

That will ensure that we can handle at least, 70 requests per second, per replica.

Disclaimer: as I said before, this is a simple approach, we are not considering several things, for example, node resources, MongoDB capacity…