Saturday, June 26, 2021

SKP's Microservices #1 - Failures, Pitfalls & Disadvantages of Microservices

Introduction to SKP's Microservices Masterclass Series

SKP's Microservices Masterclass is a Series of Articles on Microservices Architecture (MSA) - Starting with the Needs and Motivations of Microservices, Moving to the Shortcomings of Monolithic Architecture and Also Covering Microservices Design Patterns, Pitfalls of Microservices, Major Microservices Infrastructure Components - Description, Evaluation and Code Samples, Containers, Native Microservices, Cloud Platforms, Messaging Layers, Security and Logging. We will Provide the Readers with Obelix, A Microservices Architecture Reference Implementation! You May Use this as the Starting Point for your Microservices Architecture. You may Also Directly Use this as a Reference Implementation for your Microservices Strategy and/or as Code to Learn and Implement the Microservices Architecture for your Organization.

Introduction to Microservices Adoption in Real World

Microservices Adoption is almost becoming the Normal in almost all Software Product and IT Services Organizations. Hereby, I am taking this Opportunity to make you Aware of the Real-World Issues (Case Studies) that are faced by Organizations (Small, Mid, Large) of all Sizes in Migrating or Adopting Microservices. Since Learning from Other's Mistakes and Knowing the Pitfalls Upfront help us Prepare and Plan Better - This Article will be Helpful for those Planning a Shift to the Microservices Strategy Organization-Wide or simply Making a Move from Monolithic to Microservices Architecture.

It is Mandatory that all of you Skim Through The Below Articles (Titles) and then Read a Few Articles that you find are closest to your Job Function. In this Article, I am Providing only the Brief Explanation of the Main Challenges of Adopting Microservices. Also, You may use this as a Ready Reference Index for Articles Published on the Internet to Further Your Knowledge fo the Pitfalls of Adopting Microservices. 


Adopting Microservices has Been a Great Challenge for All – Netflix, Amazon, Uber and even IT Service Companies have had their share of Challenges in Microservices Migration/Adoption  – ThoughtworksCognizant, IBM, Infosys, Xebia. Few Companies have Reverted Back to Monolith after their Microservices Strategy / Implementation Failed or Did Not Yield the Desired Gains.  


[So, The Idea is that You Do Not Make the Same Mistakes, Learn from the Below and Try to Get it Right the First Time – Hence, We Need to Invest Time in Quality Research and Development to build the Initial and Robust Architectural Pieces. Also, To Create a Meaningful and a More Realistic Plan for Organization-Wide Microservices Adoption in a Software Product Company. These are also valid for Adoption / Evangelizing to your Customers in an IT Services Organization.]

Practical Challenges in Adoption of Microservices

  • Overhead
    The Setup of Microservices Requires More Efforts as there a Lot of Infrastructural Architectural Pieces as part of a Complete Microservices Architecture. For Example - API Gateway, Circuit Breaker, Discovery Server, Config Server, Health Monitoring, Vault Service, Token Service, Logging & Auditing - These are Just a Few of the Initial Infrastructural Services Required to Correctly Implement a Complete Microservices Architecture in the Enterprise.

  • High Skill
    There is a High Skill Level Require Especially in Terms of Experienced Senior Architects, DevOps and Developers who can Now Understand the Change in Development Paradigm. Also, This May Either Increase the Development Time or Development Costs or Both.

  • Complexity
    One of The Most Important Disadvantages of Microservices is the Complexity Involved. Even Though the Point is Captured in the Above 2 Points - It Still Requires a Mention on its Own. The Interplay of the Plethora of Infrastructural Services along with the Actual Application Microservices may Create Unending Cycle(s) of Complexity.

  • Redundancy
    During Development, the Architects and Lead Engineers may Need to Keep a Watch on Making Sure that there is No Duplication of Code, Configuration or any other Effort that may Already have been Developed by Another Team. This Starts Becoming a Greater Concern when there are Geographically Distributed Teams and Focussing Entirely on their Own Set of Business or Product Deliverables. Though some very Trivial Code Duplication may be Inevitable as far as Possible - Discipline will be Required to Isolate and Group Common Functionality or Code Together Correctly.

  • Data Duplication
    One of the Greatest Challenge is Data Duplication. Most Inexperienced Architects and Engineers may end up in Such Situations. It is because the Thought that was required to Build Monolith Data Access and Creation cannot Now be used for Microservices. The Mechanism to Maintain Data Integrity and Data Consistency in a Product to Make Sure that There Exits a Correct 'Single Version of Truth' Needs to be Planned, Architected and Infused into the Design.

  • Communication
    The Communication Between External World and Microservices or within Microservices may Fail. This could be Due to Multiple Reasons Including Network Failure and Service Unavailability. In any Case, The Communication Between Microservices is Critical and is Tough to Setup and Maintain. It also Requires Constant Monitoring of Health and Performance of Each of the Microservices. Partial Failures is actually an Advantage of Microservices - But Still the Monitoring, Failure Detection, Failure Handling and Simply the Internal/External Communication can be Quite an Overhead in the Implementation of Microservices.

  • Maintenance
    So, Once the Initial Part is Over - Even If the Microservices Strategy and Implementation Turns out to be a Success - It still Means a Heavy Hit on the Pocket for Maintenance. Even though all of the Current Generation Organizations Rely on an External Cloud Provider, It Still Means More Expenditure on Hardware, 'Software', Cloud, DevOps Engineers. Also, the Total Cost of Ownership of Such an Infrastructural Masterpiece is Only Bound to Increase as More Abstractions are Added Over Time - Also, The Maintenance Itself could be a Complicated Process.

FIG. 1


Articles from Online Developer Magazines (By Topic)

Challenges of Moving from Monolithic to Microservices

Top 3 Challenges of Adopting Microservices as Part of Your Cloud Migration

From Monolith to Microservices: Horror Stories and Best Practices

Advantages and Disadvantages of Microservices Architecture

Fearless Monolith to Microservices Migration – A Guided Journey

Failure and Pitfalls of the Microservices Strategy

Experiences from Failing with Microservices

How To Succeed at Failure with Microservices

How to Condemn Your Microservices Architecture to Fail

Your Microservice Architecture Will Collapse

Learn from Other's Mistakes in Microservices

5 Big Microservices Pitfalls to Avoid During Migration

Tech Talks With Tom Smith: Issues in Migrating to Microservices

11 Reasons Why You Are Going To Fail With Microservices

Failing at Microservices - Please Avoid Our Mistakes!

Planning a Microservices Adoption in the Enterprise

Pitfalls & Challenges Faced During a Microservices Architecture Implementation

4 Challenges You Need to Address with Microservices Adoption

Challenges when Implementing Microservices

How to Plan a Microservices Migration

Fearless Monolith to Microservices Migration – A Guided Journey

Real-World Case Studies of Microservices Adoption

Why and How Netflix, Amazon, and Uber Migrated to Microservices: Learn from Their Experience

Explore UBER’s Microservice Architecture

Why Microservices Fail? — Xebia Blog

Is it the Death of Microservices - Already, In 2021 ?

Microservices to Not Reach Adopt Ring in ThoughtWorks Technology Radar

The Death of Microservice Madness in 2018

Goodbye Microservices: From 100s of Problem Children to 1 Superstar

Unraveling Microservices: Higher Agility or Hype?

Conclusion and 'Gyan' for Real-World Adoption

Let us Take a Look a Uber's Microservices Dependency Graph (Source : Online Presentation by an Uber Engineer - All Rights Reserved by Uber - Solely Used to Demostrate Complexity). It Seems like an Outer Space Image from a Hubble Telescope! :-)




Idea is not to cause Paranoia or Pessimism – But to Better Estimate and Plan for Success. The Microservices Animal isn’t like our Regular Monolithic Technology Evaluations – Only  Working Together in a Focussed Way, We can Tame it Decisively. Else, We may end up in Being Unable to Achieve Desired FunctionalityCycles of ReworkPoor ArchitectureHigh Data RedundancyQuestionable Data IntegrityPerformance Worser than Monolith among Other Issues – We may even have to Revert Back / Miserably Fail.


Individually, I Have Worked in 3 Organizations where Microservices Adoption or Migration was Going On – As Far as I Know, the First Project was Shelved after about 4-6 Months – In The Second One, 'Real Microservices' Did Not Go Ahead even after about 1.5 Years (Including the Time of 6 Months After My Joining) – It was Far from a Microservices Readiness or Certification. In the 3rd Organization, where Majority or Meat of the Experience for this Article comes from - I was in charge of Technically Leading the Microservices Strategy, in the role of a Principal Architect. Until my Tenure in the Organization, We had Upfront Identified Major Issues in Architecture by Group Review Processes. We had also Improved Upon Existing Implementation by Iterating through Cycles of Infrastructure Architecture, DevOps Architecture, Technology Evaluations, Creating Reference Implementations, Deciding Release Branches, Regular Architecture Demos and Planning for Architecture Validation and Deciding to Roll out the Reference Architecture Implementation for General Developer Usage in Product Development.


Kindly Take Time to Internalize these Thoughts as Microservices Architecture is the Current Big Bet for Software Product Organizations and IT Services Organizations in 2021. It will continue to be the same for a Few Years from Now! Organizations are going to Spend a Maximum Part of their IT/R&D Budget in Adopting Microservices - So, Upfront Identification by Stakeholders will give the Bang for the Buck! 

So, Here's Wishing you a Lot of Skill, Focus, Pragmatism, Understanding, Reasoning and Continued Experience to get the Best Out of Microservices for your Organization and Needs!

[Disclaimer : Please Note that the Article Content in the Links Provided in this Article is the Sole Property of the Respective Authors and their Publishers.]