The Tao of Microservices

My book on the microservice architecture is in early release! This book is based on five years of building microservice systems, of all shapes and sizes. It is a comprehensive guide to using the architecture based on practical experience. I’ve made many, many mistakes along the way – you don’t have to!

The number one reason to use microservices is that they put the fun back into building software systems. Forget about all the serious reasons for the moment. As a matter of personal experience, the microservice architecture is just so much more enjoyable to work with. I’ve been building systems this way since about 2011. Systems big and small. I’ve tried most of the permutations. More often than not, it has all worked out wonderfully well. Each time things get better. A good idea should get better the harder you work it. I’ve worked microservices pretty hard, and I’m still working them, and I’m still coming back for more. The closer you look, the better they get. And they make you feel wonderful.

I’ve decided to write another book, despite swearing that I would never write one again. Writing is the most painful thing you can do to your brain. It forces you to think clearly! A most uncomfortable experience. The wonderful thing about putting your ideas on paper is that it starts a conversation. There is no question that a great deal of work remains to be done to refine the microservice idea. That only happens when we apply our collective intelligence to the problem. This book is a field report from the front lines of the microservice architecture. It captures an approach that has evolved from practical iteration.

At the end of 2011 I co-founded a software consultancy. I had finally had enough of the startup merry-go-round, and decided to try a different business model. As it turns out, it was a hidden passion, and we have had great success and grown quickly. Part of that success is due to our adoption of the microservice architecture (the other part is our participation in the open source community). Microservices let us deliver quickly. Fact. If you want empirical evidence that microservices actually work in practice, nearform.com is it. The great thing about software consultancy is that you get to work with so many people and companies, and so many different technical challenges, from greenfield projects to massive legacy integrations. It’s always interesting to hear how Netflix, or Uber, or Amazon are using microservices, and they have very much moved the needle (thank you guys!). But there is still the criticism that these are unicorns, special cases, and that you need the vast resources and teams of rock star coders to make it actually work in practice. The great thing about consultancy is that you can gain such a wide range of experience that you can really see how software development plays out in many different contexts. Microservices can work for any team, right from the word go.

So why are they so much fun? Because they are little building blocks that snap together easily and let you build big systems without melting your brain. So much of the pain of software development comes from the mental effort required to keep lots of spinning plates in the air. Our language platforms have too many features, and too much power. This power is great, and fun to code, when the code-base is small. As soon as it reaches enterprise-grade applications, with massive code-bases, it becomes very not fun. This is called technical debt. It sucks. Technical debt is the reason for lost weekends, death marches, and failed projects. Microservices are the antidote to technical debt. They make it very hard to take out that loan. They are small, so you can’t shoot yourself in the foot.

Microservices work because they give you a scalable component model based on the principle of additivity. Need a new feature? Write a new microservice! The immediate criticism is that with so many moving parts, this is a nightmare to deploy, and impossible to understand. Well certainly, if you think microservices are just a Service-Oriented Architecture, with more and smaller services, then you’d be dead right. That would be a disaster. The key to making it work is to stop obsessing about the services, and turn to the messages between services. The messages can be catalogued. They form a domain language. They can be directly connected to the business requirements. The desired behavior of the messages, and their interactions, can be declaratively defined (pattern-matching is a good approach). Once you have the list of messages, you group them into services. The number of service instances, and the nature of their deployment, and even the transport mechanisms that you use to move messages, are all ultimately infrastructural issues that can be solved with deployment automation and management (you should be doing this anyway, even for monoliths!). They are not fundamental weaknesses, no more than the need to compile a high-level language into machine code is a weakness – it is just a practicality.

The book has two key aims. First, to give you the practical and theoretical tools to design, build and deploy microservice architectures that work, and that give you the benefits of rapid development, flexibility in the face of changing requirements, and continuous delivery. Second, you need to understand the trade-offs of the architecture, so that you can understand the advantages and disadvantages of choosing to use microservices. Your own situation is always relevant. It is your job to make the decision. The problem with all the noise, all the blog posts for and against, is that it is very hard to get a clear understanding of the consequences of your decisions as a software developer. This book is a field report – a concise summary of my five year journey, and the place where I am now. Compress my five years in 5 hours of reading, and make your own decision about microservices.