An engineering strategy requires context specific metrics
As part of my Design and execute an engineering strategy blog series, I didn’t emphasize enough the importance of metrics, and which role they play.
Metrics helps you on:
Aligning on what you want to impact the most.
Connect the technical initiative with the business impact.
Know when you achieved your goals, and you need to move into the next thing.
That’s why I dedicated a full blog on how to combine Engineering Strategy with the North Star framework, borrowing the idea from our product colleagues.
Yet, I consider it is great to dedicate another post to emphasize how the metrics are essential for a successful engineering strategy.
Misunderstandings
-X: Are DORA metrics good to measure the engineering strategy?
-Aleix: No, or only in very concrete cases. They help us as guard rails to know that we aren’t getting out of track.
DORA as an engineering strategy metric
DORA metrics helped us all to understand how we are doing compared to other companies in the field. They are a great proxy metric to measure delivery performance.
Yet, they aren’t a replacement for context specific metrics to understand the engineering strategy impact.
I might consider DORA metrics to be suitable as the main engineering strategy metrics when they are in the low performance stage, and you need to improve them.
Once the DORA metrics are good enough, don’t use them as guiding metrics. Just to keep yourself on track.
No need for measuring, just execute
Designing an engineering strategy is hard, investing time on making sure to measure the progress might seem like an over-investment, but it’s not. It is crucial to execute well the strategy.
Without metrics, you cannot:
Keep people and teams focused on the metric that matters.
People might start coherent actions that don’t impact the key metric.
People will not see how they work impacts the strategy.
You don’t know when you addressed the high stake challenge, and you should move into the next. When do you know you finished?
Your metric will act as an information radiation, and keep people focused on the engineering strategy objectives. Plus, helping them knowing when to stop, and go for the next big thing instead of over improving an area of your organization.
Measuring is hard
Yes, measuring is hard. But we don’t need to create complex ways to measure the engineering strategy progress.
Sometimes, with an Spreadsheet is enough.
Do we need to migrate those services into the new way of deploying?
List all the services on a spreadsheet and use some colors to mark progress.Should we replace Kinesis by Kafka?
List all the services or use cases pending to be migrated in a speadsheet, and keep the progress visual.Should we improve the amount of bugs related to this system?
Add some tags to Jira tickets and create a report there. Or connect those with Notion with a NoCode integration like Make or Zappier.
Avoid involving the data team when possible, you don’t want to add more pressure to teams to execute the strategy. Make it easy.
Engineering strategy metrics life cycle
The engineering strategy metrics, because they are context specific, they tend to not last too much. Usually, they life-cycle is as long as the engineering strategy is valid.
Once the engineering strategy is done. Either you no longer track them anymore, or you move them to supporting metric to be sure you don’t go backwards on future initiatives.
Deciding the right metric
A good engineering strategy metric is the one that:
It is connected to business/user value, and it is aligned with the business strategy.
Improving the metric has a positive outcome for the business and engineering.Acts as a proxy of engineering strategy success.
Are we impacting the strategy as we expect?
Deciding the right metric is hard, that’s why I will share some real world examples that I have been involved with.
Fintech - Not releasing a product
Full engineering strategy story here 👇
Business Challenge
We aren’t releasing the products after one year of development. We need to be sure the product is viable, and verify product market fit.
Engineering Strategy
Full stack teams to remove handovers between teams, and simplify the architecture from ports and adapters to CRUDs.
Engineeering Strategy Metric
Lead time from initiative defined to first user.
Fintech 2 - Integrating new banks
Full engineering strategy story here 👇
Business challenge
For each new client that we onboard, and we don’t have an integration with their bank, we need to create the integration for that bank.
Engineering strategy
Build a platform team to create a compeling experience to integrate new banks.
We will create an isolated account integration platform with the goal of creating a self-served integration for Account Opening Team. It is important to build an internal product first, before aiming to expose it to the end client.
Engineering strategy metric
Lead time to integrate a new bank from a request from the client is done until it performs the first transaction.
Supporting metrics
The supporting metrics are those that apply to each engineering strategy regardless of the context. They help you to know that you aren’t getting worse on them.
DORA metrics
Once you have a good enough DORA metrics, you want to be sure that you aren’t becoming worser overtime. Use the DORA metrics to keep on the right track.Team Cognitive Load
You can understand the impact of the engineering strategy though team cognitive load. If the strategy starts adding unnecessary cognitive load to people, it might prevent the engineering strategy to succeed, and you have to take action.eNPS
Use Employee Net Promoter Score to understand if people feel more engaged with the company and see their pains are also taking into account in the strategy.
Do you need help?
I help organizations to design and execute an engineering strategy, sometimes you just need 30 min to clarify some doubts on which is the best context specific metic for your engineering straregy.
You can book 30 min with me for free in the next link.