Let's Not Demonize Technical Debt - II

In one of my previous articles, Let's Not Demonize Technical Debt, I wrote about the genesis of technical debt and why it gets a bad rap. I explored the scenarios that lead non-tech companies to accrue massive technical debts and, ultimately, how they fail to use it as a strategic leverage tool.

Many of you wrote to me asking for more - insights, actionable ways to tackle tech debt and practical measurement processes. In this piece, I am sharing various types of debts, measurement dimensions, leading and lagging indicators, and tools and techniques that organizations have successfully deployed to manage and pay off debts. In the next part of this series, I will share notable case studies that will make this real and bring it together.

I hope you enjoy reading this article and find it insightful. Leave me a comment or email me your thoughts and ideas on this subject.

Quick Recap

In my previous article, I borrowed concepts from the basic financial debt management principles to establish five ways to get a grip on this matter:

1. Adjust the attitude towards debt: Define and exemplify tech debt for your context - what constitutes it, how you recognize and classify it
2. Not all debts are bad: Delineate between good debts (prudent and deliberate) and bad debts (reckless and accidental); use good ones for strategic leverage while avoiding bad ones.
3. Know your payback plan: Before you take on the debt, analyze how and when you plan to pay it off.
4. Keep your books updated: Measure and track debt-to-asset ratio metrics on product/ app/ portfolio health dashboards.
5. Communicate how debt accrual is hurting your goals: Treat this as a business issue - describe the negative impacts on product performance, quality, team morale, and to organization's top-line and bottom-line.

Let's dive in...

I. Tech Debt Types

Over the years, scholars, software researchers, and scientists have tried to establish an ontology and common vocabulary for this concept [1].

On the lower end, it can be a small area of code that requires refactoring, libraries that need upgrading, or unit testing that needs fixing. On the higher end, technical debt includes reengineering complex monolithic applications, porting outdated web service protocols, consolidating multiple platforms to one standard, modernizing infrastructure, introducing observability practices. The worst type of technical debt is a ‘burning platform,’ meaning a platform with recurring outages and incidents that impact the business. [2]

Aligning on a taxonomy and understanding the relationship between these categories is essential as we delve into measuring and monitoring debt accumulation. After poring over many publications, I created a simplified view that covers the five most common types of debt.

Type 1: Design Debt

• Description: Problems within software architecture; can be application/system-level, solution-level, and Enterprise-level architecture.
• Symptoms/ Examples: Violation of modularity principles and domain boundaries resulting from lack of Architecture Standards and Guiding principles.  Commonly seen example: continual extension of the software to support adjacent and/ or orthogonal use cases, ultimately veering too far away from intentional design, inducing maintenance complexities and operational dependencies

Type 2: Code Debt

• Description: Poorly written source code that is illegible/ unexplainable and is rife with issues such as violation of object-oriented programming principles
• Symptoms/ Examples: Lack of comments for readability, overuse of chained functions, nested conditional statements, code duplication, accidental coupling of modules via bad imports and bad project structure, and behavioral inconsistencies across methods.

Type 3: Quality Debt

• Description: The software does not meet the  functional and non-functional business requirements resulting in subpar utility and user experience
• Symptoms/Examples: High volume and severity of defects, frequent incidents and outages, longer resolution times

Type 4: Infrastructure Debt

• Description: When software or hardware infrastructure is not at par to provide the required business agility and speed to market
• Symptoms/Examples: Deferred software upgrades and immature deployment infrastructure (lack of environments and CICD automation) leading to longer lead times

Type 5: Knowledge Debt

• Description: When information on how the software works is concentrated in small pockets and cannot be easily shared and scaled
• Symptoms/Examples: Inadequate documentation, elongated onboarding time for new resources to be fully productive, and technical know-how of bespoke apps concentrated within a few SMEs creating a talent risk (retirement, churn, attrition)

II. Measurement & Metrics

This article focuses on Design, Code, and Quality debts. These can be measured quantitatively, with a handful of metrics that are leading and lagging indicators for debt accumulation.

Infrastructure and Knowledge debts can be assessed qualitatively and are addressed by defining architecture standards and enforcing them via optimal governance and by maturing the IT infrastructure management capabilities.

Type 1: Design Debt

Measurement and Pay-Off Mechanism

Portfolio Health Assessment (bi-annual/ annual process): Conducted by senior engineering and product leaders resulting in a current health disposition and proposed path to getting healthy. Most often, this type of debt needs intensive design and development activities and cannot be paid by simple code refactoring sprints. The explicit funding commitment is procured for paying this debt off, as it can take 6+ months of effort.

Metrics

Though there are many dimensions, in my experience following 3 carry the most weight in determining asset health disposition:

1. Business Agility Score: the degree to which the application can adapt to changing business requirements
2. Technical Currency Score: degree of alignment with industry and enterprise technical standards
3. Maintainability Score: degree of effort needed to keep the operations running smoothly

All these metrics are lagging indicators of tech debt.

Type 2: Code Debt

Measurement & Pay Off Mechanism

Lead Engineers, Tech leads, and Application Owners measure Code debt regularly. Highly mature companies allocate a percentage of capacity (15-20%) to maintenance efforts, including code refactoring. In special situations, companies invest in designated "refactoring sprints" where new feature work is paused until the Tech Debt is paid off. In emergency triage situations, a special team can be tasked to pay off the tech debt. However, this is an ineffective and unsustainable practice and should be reserved for exceptional cases.

Metrics

1. Code Ownership: This is a leading indicator of Engineering health. Maintaining high coding standards in a tight group that is well-informed about their part of the codebase is easier. On the flip side, orphaned code that no one wants to touch is a sign of bad technical debt. Unintentional code debt will likely accrue as a side-effect of incomplete information and diluted ownership.
2. Cyclomatic Complexity: This measures "the amount of decision logic in a source code function."[3] Simply put, the more decisions code has to make, the harder it may be to test, maintain, and troubleshoot. The most effective evaluation of code complexity is a combination of code size (lines of code within a function) and cyclomatic complexity. [4]
3. Churn: Measuring Churn (repeated activity) finds surface areas of change, aka the files that produce most bugs. In a growing system, it's typical for some files to be the hubs and remain active as new features are added. However, it's a good idea to refactor and rigorously review, test, and QA these files and churning areas. In contrast, for a  mature system, churn helps identify the components to break down to minimize the surface area of change and optimize stability. For systems tagged as Legacy, churn helps identify the active components so the team can plan how to phase them out.
4. Cohesion: This is a trailing indicator of highly cohesive components. Understanding how strongly the components within an application are coupled informs code refactoring efforts.
5. Deployment Success Rate: If the number of failed CI/CD builds increases, it's a strong indicator of instability in your codebase. This issue can be related to both design and code debt.

Type 3: Quality Debt

Measurement & Pay Off Mechanism

Quality debt is measured and managed by QA and Product Management in partnership with engineering leaders. While quality metrics at the application/ product level are tracked and reported daily at most organizations, historical trends and pattern analysis at the aggregate level are often missed. Quality debt management requires high maturity of Engineering and QA discipline.

Metrics

1. Code Coverage Percentage or Code Coverage per feature: A declining percentage is a red flag for a growing tech debt.
2. Defect Trend/ Defect Open Rate: If your team is beginning to find more and more bugs weekly or monthly, it indicates a quality decline. Therefore it's a great metric to detect a growing tech debt.
3. Performance Monitoring: Bad response time will result in UX/ Usability issues. Performance degradation is a solid indicator of increased technical debt.

These are lagging indicators for Software Quality; however, these are leading indicators of growing Code Debt and Design Debt.

Other Supporting Metrics that indicate growth in tech debt:

1. Mean-time-between-failures/ Mean-Time-to-Resolve:  An increase in incident frequency and resolution time points to potential tech debt growth and the need for refactoring to address the root cause
2. Feature Throughput: If the number of days to push a new feature to the main branch is increasing over time, there could be a reason to investigate the root cause.
3. Time to Value: This measures the release performance and refers to the time taken before a complete written software build is released into production. The delay may be caused by several factors, including infrastructure resources and automation capabilities available to test and process the build and the governance process necessary for the final release.

III. Tools & Processes

• Static Code Analysis: Tools such as SonarQube, CodeScene, and CodeClimate [5] are used to automatically analyze source code and detect code quality and security issues during development. These tools use quantitative data to help developers identify hotspots in the codebase likely to have technical debt.
• Code Debt Systematic Tracking: Tools such as Stepsize [6] allow engineering teams to track technical debt in the code editor and tag it to issues in the issue-tracking system. Developers already spend most of their time in the editor, so it's the best place to track and report technical issues.
• CI/CD Analysis: A continuous integration (CI) tool can guarantee code quality by enforcing coding standards and performing integration and end-to-end testing. Leading DevSecOps platforms offer analytics capabilities to measure code ownership and stability, team velocity, deployment success rate, pipeline efficiency, and incidents management metrics (mean-time-between-failures, mean-time-to-respond/repair).
• Tightening the feedback loop between shipping and running: Pulling engineers from feature work to be on-call rotations for fixing, refactoring, and automating away the debt develops a healthy tension between shipping new code and supporting and improving what already exists. If there are recurring incidents, that points to an accountability issue that might need a change in ownership and operating model.
• Tech Debt Backlog Tracking: Anything you recognize as technical debt, track it as another feature to be included in your normal agile process. Adding it to the backlog/issue tracking system (versus documenting it in unstructured files, confluence, docs, and slides) makes the tech debt measurable and visible on dashboards and stays front and center of engineering decisions.
• Strengthening Business and Digital Acumen: Addressing technical debt may be the first thing to get deprioritized in the pursuit of business goals, but engineering managers must change that narrative by tying debt pay-off with business metrics, like turnaround time, performance,  quality, and customer satisfaction, and talent retention. A strong partnership with Product Managers/ Leaders is a crucial enabler for learning storytelling and garnering support for tech debt remediation.

In Closing

Technical debt is but a legacy of decisions made till date. Doing things the right way is always going to be difficult. Making something elegant and simple takes more thought processing than complicating it. Such is life and engineering is no different. It requires a sense of responsibility to take on new debts; and calls for business savvy and strong political will to garner support for paying off the debts inherited from your predecessors. So, what kind of legacy you want to leave?

Technical debt is inevitable, and being proactive about reducing debt over time is the best way to tackle it. Although some scenarios warrant one-time big-bang debt remediation projects, they fail to deliver sustained outcomes. Being proactive implies establishing policies and processes and implementing tools to amortize the cost of reducing debt over time.

Since you have reached the end of this post, I assume you like my content. If so, click here to subscribe to my newsletter and get exclusive content from me.

Disclaimer

I am not compensated by companies for promoting, reviewing, or recommending their products or services.

References

[1] N. S. R. Alves, Leilane Ferreira Ribeiro, Vivyane Caires, T. Mendes, R. Spínola (2014). Towards an Ontology of Terms on Technical Debt (2014 Sixth International Workshop on Managing Technical Debt)

[2] https://www.infoworld.com/article/3660632/you-re-thinking-about-technical-debt-all-wrong.html

[3] Structured Testing: A Testing Methodology Using the Cyclomatic Complexity Metric, NIST Special Publication 500-235

[4] Cyclomatic Complexity and Line Numbers

[5] Static Code Analysis Tools Overview

[6] Document and fix issues in your codebase with Stepsize

[7] Rosenberg, L., Hammer, T., Shaw, J. (1998). Software Metrics and Reliability (Proceedings of IEEE International Symposium on Software Reliability Engineering). Retrieved May 14, 2011, from Penn State University website

[8] https://stepsize.com/blog/8-top-metrics-for-measuring-your-technical-debt

[9] https://www.stepsize.com/blog/complete-guide-to-technical-debt