How to measure tech debt?

It is quite important to measure tech debt as thing you don’t measure typically do not get managed and if we look at our research data it seems that managing complexity and technology debt is a high priority for Enterprise Architecture.

A quick look at consumer finance

Before we talk about technology debt let’s talk about consumer debt for a moment. If a person is buying stuff like computer games with money they don’t have they get a loan from their bank. Going forward they are paying interest and are repaying the loan. This can lead to a difficult situation as they might have spent so much money on computer games that they lack the money for food, rent or other essentials.

If we want to measure consumer debt we have these options:

• We measure the root cause of the debt – in our example how many computer games are on the shelf?
• We measure the interest payment – how much money goes towards the debt that can not be used otherwise?
• We measure the negative effect of debt – how limiting is the debt, i.e. how reduced is the budget for e.g., food?
• We measure the debt repayment – how much are we reducing the debt?

How to apply this to technical debt?

The story of technical debt is actually quite similar to the consumer finance example above: Someone made a mistake in the past and e.g. has put all the software on a monolithic mainframe platform. The impact of this decision is a few decades later still a substantial amount of money flowing to the mainframe vendor of choice, it is limiting the ability to create new features and only by refactoring one can get rid of the technical debt.

As in consumer loans these are the factors we can measure:

• Root cause: How many mainframes are still in the data center?
• Interest payment: How much is the monthly IBM bill for the mainframe?
• Negative Effect: What is the negative effect of this platform on e.g., the cycle time?
• Debt Repayment: How much effort goes into refactoring towards open systems?

If you are looking for the perfect way to measure these factors you can stop reading as I won’t be able to provide you with the right answer. And from my point of view you should not try to look for the perfect solution. Perfect is the enemy of good very much applies to measuring technical debt. Rather than developing the perfect solution aim for a good proxy that focuses on the measures that you have easily available and those that matter in your organization.

Tangible examples

Measuring the root causes is most of the time the easiest approach as one can pull the numbers from the CMDB or simply go to the data centers and count machines. Here are a few examples for metrics that are focusing on the root causes:

• What percentage of the applications is running on mainframe?
• What percentage of the development is done on systems that are out of maintenance?
• What percentage of the OS images no longer gets security patches?
• What percentage of the projects are using tech stack components that are not part of the target landscape?
• …

The interest rate is often paid with money but it can also be paid with internal time, higher risk levels or other negative effects. Here are some examples for metrics that focus on the negative effects or other forms of “interest payment”:

• How much effort goes into maintaining customization and how high is my RICEFW score for the SAP systems?
• How much time goes towards integration efforts vs. developing value adding features? (see this article for some benchmark numbers)
• What is the effort that goes towards change vs. keep the lights on?
• What is the average velocity of the development teams (measured e.g. in function points per person month)?
• What is the average cycle time? What is the release cycle?
• How many severe incidents have been reported?
• How many bug have been found after release into production?
• …

An last but not least one can also measure debt repayment, i.e. refactoring and transformation efforts that should address the root causes:

• How much mainframe capacity has been reduced?
• How much code is migrated to the target platform?
• How many new, consumable interfaces have been created?
• What percentage of new project is compliant with the target tech stack?
• …

Get started

All of these are just examples. You have to pick those that are representative for the kind of problems tech debt is causing for you. Is it the long cycle time caused by old Fortran programs? Is it the security vulnerability by Windows NT? Pick those that matter in you context and start measuring.

Most likely you want to combine several of these measures into an index that adds weights to the individual elements. You can then define what good looks like for this combination of measures.

Let me know in the comments how you measure tech debt and how the rest of the company reacts to these measures.