3 September 2012

How to manage down the payments on your technical debt

Every software project accrues technical debt over time. It’s a great metaphor to describe the corrosive effect of quick and dirty design decisions on your productivity.

Like a financial debt, a technical debt incurs interest payments in the form of reduced productivity and you have to spend development time to clear the account. If you ignore the interest then it can accrue to the point of technical bankruptcy where you are unable to safely make any further changes to a system and development grinds to a halt.

On the face of it, the solution to technical debt is pretty simple. You have to identify it, work out how to pay off the most profitable debts and control the accumulation of new debt. The difficulty lies in properly identifying and measuring technical debt so you can start to manage it.

Identifying technical debt

Technical debt is an over-used term and can become a catch-all phrase for any difficulties associated with normal development. People often mistake complexity for technical debt. Just because something is difficult it doesn’t necessarily mean that it’s somehow being executed badly – it could just be hard.

It’s important to identify exactly what we mean by technical debt, as opposed to manageable complexity. Ward Cunningham, who first coined the term, suggested that technical debt encompasses any deferred work that couldn’t be included in the current development cycle. This doesn’t necessarily cover all the bases as it implies that technical debt is always the result of conscious decisions.

Although technical debt often arises from conscious, pragmatic decisions it can be accidental or caused by external factors, such as changes to related systems or external APIs. It can also be triggered by internal issues within a team where negativity and cynicism starts to undermine development or cultural factors such as an absent sense of code ownership.

Another major cause of technical debt is time, i.e. It. Just. Happens. Technologies change, frameworks mature and APIs get deprecated. People come and go from the development team. Priorities change and things get forgotten about. A long-serving code base will inevitably accumulate some form of technical debt – it just needs to be identified and managed.

A more comprehensive definition might be to regard technical debt as anything that has a negative impact on productivity that can be eliminated through some work. This is quite a wide definition, but technical debt is quite pervasive. After all, once you start looking for technical debt you can find it pretty much anywhere – examples of where to look include:

  • The clutter of incomplete development tasks such as failing tests, ugly code that is difficult to read and outstanding bugs.
  • Structural issues such as missing development infrastructure or incomplete process.
  • People-related issues including knowledge silos and unproductive developers.
  • Low priority requirements that never make it up the priority list.
  • Change that happens outside the immediate system, i.e. changes to related systems, tools or operating systems.
  • Cultural issues such as denial or cynicism.
Once you start identifying technical debt then you should start to capture it. This doesn’t have to be a tortuous process – just a list will do. if you’re working in an agile environment then it can be captured in a backlog. The important thing is to  make a continuous effort to identify and catalogue your technical debt as a first step to dealing with it.

Measuring technical debt

Despite the neatness of the metaphor, the notion of technical debt does begin to break down over measurement. Where financial debt can be given monetary value, a technical debt affects productivity which is notoriously difficult to measure. Traditional productivity metrics such as code output, relative velocity or function points do very little to provide insight on the true impact of technical debt.

A more useful way to measure technical debt is to focus on individual aspects of debt and identify their individual cost. This can take a number of forms, such as opportunity cost, development time or learning curve. The point is that it’s much easier to attach monetary value to technical debt when you focus on smaller, more management items.

Once you start identifying and quantifying individual aspects of technical debt so it becomes easier to construct a case for tackling the debt. It helps to put technical debt into a clear commercial context so you can actually start dealing with it rather than labouring under a vague and ill-defined overhead.

Ultimately, the management of technical debt comes down to money and risk. Sometimes it really isn’t worth dealing with an aspect of technical debt. It may be something you would do differently if you had your time again, but it isn’t killing the system. Not like that unfixed security vulnerability that you keep leaving until the next sprint.

Managing your payments

As with financial debt, there are no quick fixes to technical debt and it requires persistence and discipline to address. One difficulty is in winning support for dealing with technical debt as it rarely gives rise to the kind of functional improvements that stakeholders seek. After all, technical debt is generally caused by leaving out work that was originally regarded as low priority.

An iterative approach can help with stakeholder management as it allows you to demonstrate value through small, incremental successes. You identify the most profitable items in your catalogue of technical debt and focus on them to demonstrate progress one at a time. The trick is to demonstrate measurable progress. You don’t have to completely solve the problem or even eliminate all the debt that it accumulates, you just have to deliver some measurable impact so you can establish momentum and win support for future effort.

Addressing existing debt is only part of an overall strategy for debt reduction. You also have to prevent new debt from appearing which can just add to your overall interest payments. There is no magic wand here as you just need diligence, ownership and governance to ensure that short-cuts are minimised and further debt does not creep in from the corners.

An on-going process of debt reduction can also help to prevent the accumulation of new debt by raising greater awareness of the long-term impact of technical decision making. One problem with measuring productivity is that it can take a long time for the impact of technical decisions to be truly felt. A development culture that regularly tracks and assesses technical debt is more likely to make informed decisions around how short-term decisions can affect long-term productivity.

Filed under Architecture, Development process, Strategy.