Simplifying a Ports and Adapters architecture and remove anti-corruption layers made sense from a sociotechnical perspective
Fintech Engineering Strategy. Post IV
The Fintech post series aims to share my personal experience as an engineer manager and later on as head of engineering, which were the challenges, the decisions, and the good and bad outcomes they had. The content has been adapted to keep the decisions without disclosing internal information.
Fintech Engineering Strategy Post Series
Post IV. Simplifying a Ports and Adapters architecture and remove anti-corruption layers made sense from a sociotechnical perspective. [This post]
I’m concerned about the amount of misunderstood ports and adapters architectures (from now on P&A arch.) out there, more specifically on the decision making process behind them.
The intention with this post is to give you tools and arguments to decide properly which architectural choices do you have and, instead, opt for an evolutionary approach when needed.
Architectural choices are sociotechnical systems decisions.
I briefly introduced the decision we took of replacing Ktor by Spring Boot in this post, yet, I think I could go more into the details and nuances of the decision.
If you want the short version of the full post, you can check this Tweeter thread.
In the thread I say replacing ports and adapters in favour of MVC+S. Indeed, I would say that what we did was a very thin ports&adapters where the ports and adapters were the ones that Spring Boot provides out of the box with spring-boot-web.
The thing is that a ports and adapters leveraging Spring Boot defaults looks like an MVC+Service, or 3'-layered architecture with an Application Service.
Ports and Adapters architecture isn’t a best practice nor a sign of quality by itself. Indeed, it can even show a poor situational awareness and imply a high accidental complexity and incurred cost to the business.
This is due to it is misunderstood and each implementation is different. The possibility of introducing accidental complexity is high due to it relies a lot on the developer criteria. And we have the tendency to complicate things, right?
If an architecture rely on the developers to keep it simple, it will tend to get complicated rather than the oposite.
I don’t mean that we shouldn’t write sustainable code with a good test base and good practices like observability. I mean that applying all the “popular best practices” without understanding how they apply to our context is a receipt for failure. Applying patterns without the situational awareness shows a lack of seniority or a desire for a CV oriented career.
Disclaimer: I did that mistake, and I took a long time to realize that the business is who keeps paying the consequences.
Learning them and applying specifically for the context at hand is what matters.
Today, I’m explaining you why we made some contra-intuitive decisions such as:
Migrating from ports and adapters full decoupled from the framework to ports and adapters coupled to spring boot.
Moving from Ktor and Exposed to Spring Boot.
Moving from anti-corruption layers to decouple from external systems to conformist (accepting the upstream model).
Coupling to Spring Boot and to upstream systems.
Introduce a Platform as a Product in the tribe lead by a Platform Team.
Which helped us to achieve stuff like:
Removing +1.000 LoC cross multiple services.
Removing several cross layers duplicated tests.
Reduce the cycle time and improved the lead time for features.
Improve the reliability of the systems.
Reduce the cognitive load of teams
Improve the onboarding time from 4-5 months when people felt productive to less than two months.
As side-effect, we made decisions based on the learnings we had that impacted the performance review, the career leader definition and the hiring process. In specific, it affected the senior+ developers job expectations. I’m also explaining why and what we did here.
We just merged two teams, and we started to gain team momentum. People were doing pairing, started to know each other, and we were progressing from forming to storming based on Tuckman's model.
When things start to become less chaotic at human level, you can slow things down to understand the technical part.
At tribe level, we had an architecture like this:
Warning: Notice that the diagram now it changed compared with the 2nd post series. A new team was created to handle the shared services. This is team was named core team. Later I share why the name core was bad, and why the approach had a good intentions yet was a bad implementation and why evolving into platform made sense.
If we look at each service, the architecture was ports and adapters. If we make a diagram to show the size of each layer, it would look like this:
We can notice a couple of things:
Services have few domain logic.
Services have a lot of network logic.
Domain logic poor-network rich logic service. And it made sense, we chosen an off-the-shelf solution that would handle a huge part of our business logic as it wasn’t a business differentiation but needed to run our business. So, we were calling a lot those external systems to handle the majority of use cases.
Two main questions raised:
Why ports and adapters if we mainly pass payloads around and we have very specific business logic?
The product complexity isn’t that high. It looks like an “state machine” plus smart component that connects specific.
But we are connecting to a lot of other systems that we don’t know if they will change the API or something. So, we want to protect ourselves from future changes regardless of the system. We apply this to all systems we integrate with.
Why that many ACL? From what we are protecting from?
When a team requires a change on the shared services owned by core team, it impacts all the other teams and we need to protect ourselves from those unanticipated changes. That gives us as a team the possibility to be autonomous to change things without breaking the others systems.
Each time we change a property in a system, we need to update the ACL of other systems we control. Usually, a change on one shared service, implies a change at least to two more services, ours and the BFF (backend for frontend), which we also did with Ktor and Ports and Adapters.
The conversation started at architectural level, but the problem wasn’t there. Focusing on the technology ignoring the human aspect of the problem would let us on the wrong direction.
The architecture and approach was a reflect of how the organization introduced change, who made decisions, and different incentives mechanisms.
In order to understand what was causing the outcome, I focused on the people.
Context Mapping from Domain-Driven Design strategic patterns.
Core domain charts to understand the domain complexity.