The Technical Debt Avalanche: When Most of Your Engineering Time Goes to Firefighting

A six-year-old integration failed at 9:10 on a Monday morning. The alert said the payment service was unavailable. The service was running. The failure sat in a batch job that translated data between two systems, had no owner in the current org chart, and depended on a credential nobody wanted to rotate during trading hours.

By lunch, four engineers had paused roadmap work. One traced logs, two reconstructed the data contract, and one coordinated with operations across sites. The patch took less than a day. The business paid for it with a week of displaced product work, a postponed release, and a new instruction to “be careful around that integration.”

That is how technical debt becomes an avalanche. It does not begin with one bad function. It begins when every shortcut creates another dependency, every incident consumes planned capacity, and the team starts protecting the system instead of improving it.

This guide sets out a practical refactoring strategy 2026 for CTOs, CIOs, engineering leaders, and operators across Singapore, Australia, Hong Kong, and the wider APAC region. The position is direct: do not start with a rewrite. Measure the firefighting ratio, isolate the debt that threatens revenue or control, and repair it inside the delivery system that already serves the business.

How debt accumulates in a fast-growing product

Technical debt accumulates when short-term delivery decisions create future rework, operational exposure, or change resistance faster than the team can repay them. In a fast-growing product, the debt compounds through interfaces, undocumented rules, duplicated workflows, and ownership gaps, not only through old code.

We usually see the first layer during growth: a team hardcodes a site-specific rule to meet a launch date; an operations analyst maintains a spreadsheet because the core system cannot support a new exception; a release skips automated regression coverage because the test suite is unreliable. Each decision can be rational in isolation. The problem is that the exception becomes a dependency and the dependency becomes part of the operating model.

The 2026 industrial case study published in the Journal of Systems and Software examined four international companies and interviewed 24 experts. Its findings identified five non-technical aspects of technical debt in large agile environments: social dynamics, process, people, documentation, and requirements. That matters for a multi-site operator because a code fix cannot repair a missing owner, conflicting requirements, or a release process that nobody trusts.

We classify the accumulation into four connected layers:

• Code debt: duplicated logic, hardcoded values, weak modularity, and tests that do not protect business-critical behaviour.
• Architecture debt: boundaries that prevent independent change, integrations that rely on hidden assumptions, and infrastructure that cannot scale with demand or operational complexity.
• Knowledge debt: undocumented rules, unclear ownership, and decisions held by one engineer who is repeatedly pulled into incidents.
• Control debt: weak access management, incomplete audit trails, untested recovery procedures, or release evidence that cannot support a regulated operating model.

Faddom’s December 2025 explanation describes technical debt as the future cost of choosing an easier solution now, with interest appearing as rework, maintenance, bugs, and slower change. That definition is useful, but leaders should add one operational test: if a system requires the same people to explain the same hidden rule every time the business changes, the debt is already consuming capacity.

A custom software development company can make this worse when it treats the request as a feature queue and hands over code without preserving context. A delivery team that owns the roadmap has a different obligation: it must leave behind clearer boundaries, repeatable tests, and an operating record that lets the next increment proceed without reopening the same investigation.

Measuring the firefighting ratio

The firefighting ratio is the share of engineering capacity spent on unplanned operational work, urgent defects, recovery, manual intervention, and repeated investigation. It is more useful than a raw incident count because it shows how much planned change the business is losing to the current system.

Start with four weeks of actual work records. Do not estimate from memory; incident weeks distort perception, while quiet weeks hide recurring manual work. Tag each engineering activity against the following categories:

• Planned product delivery: roadmap features and agreed improvements.
• Reliability and control: monitoring, recovery, security, compliance evidence, and preventive maintenance.
• Firefighting: incidents, emergency fixes, production investigation, escalations, and urgent data correction.
• Debt repayment: refactoring, test coverage, documentation, dependency replacement, and boundary work.

Calculate the ratio as: firefighting hours divided by total engineering hours. Keep reliability work separate. A planned resilience improvement is investment; an engineer restoring a failed process at 2 a.m. is firefighting. Combining them produces a flattering number and a bad decision.

Track the cost in operational terms, not only engineering hours:

• Number of incidents that require the same specialist.
• Hours between detection, diagnosis, workaround, and permanent repair.
• Roadmap items displaced by urgent work.
• Manual reconciliations or site-level workarounds created after a failure.
• Release rollback, defect escape, and support escalation patterns.
• Debt items that recur after a patch because the underlying boundary remains unchanged.

The technical debt cost is the combined value of lost roadmap capacity, incident response, delayed operational change, additional control work, and the risk carried by unresolved failure modes. We do not recommend turning every item into a speculative revenue estimate. Use evidence the business can audit: hours, delayed releases, affected sites, failed controls, and repeated incidents.

For a Singapore engineering team, the comparison with added delivery capacity must include more than base salary. The fully loaded in-house senior engineer model includes base salary plus 17 percent CPF, roughly 20 percent recruiting fee, and a three-month ramp before the first shipped increment. A delivery-owning pod should be judged against the outcome it takes responsibility for, not against a salary line.

The table is not a universal procurement answer. If the work requires a permanent domain owner embedded in executive decision-making, hire in house. That is the right choice when the role will shape the company’s long-term technical direction, hold irreplaceable institutional knowledge, and remain fully occupied after the immediate debt programme ends.

For a constrained roadmap, a dedicated pod can be the more honest intervention. OmniStack’s model places engineers on its payroll and on the client roadmap, with developers, QA, and UX working as a continuing team. That distinction matters when the failure is a missing delivery system rather than a missing pair of hands. See how a dedicated engineering team is structured when you need to compare delivery ownership with rented headcount.

Debt triage: what to fix, what to isolate, what to leave

Not every debt item deserves immediate repayment. The right triage decision is to fix debt that threatens revenue, safety, regulatory control, or the next material business change; isolate debt that is risky but bounded; and leave debt that is stable, understood, and cheaper to carry than to remove.

The common market answer is to rank technical debt by code quality. We reject that as the primary method. A badly named module that nobody changes may be harmless. A small undocumented rule that controls settlement, inventory, or site opening hours may be the highest-risk item in the estate.

Use a decision record for each candidate item. It should identify the business process, affected systems, owner, failure mode, evidence, and next change that the debt blocks. The record should answer one question: what becomes harder or less safe if we leave this untouched for another quarter?

For financial services, the triage threshold is also a governance question. MAS TRM expectations for Singapore financial institutions and APRA CPS 230 obligations for Australian entities place attention on technology risk, operational resilience, third-party arrangements, and accountability. The relevant question is not whether an external team wrote the code. The accountable institution still needs to know who owns the service, how risk is assessed, how incidents are handled, how access is controlled, and how continuity is tested.

A delivery partner can own execution, but it cannot erase the client’s regulatory accountability. Put the ownership model in writing: who approves architecture, who accepts residual risk, who can deploy, who receives incident notification, who maintains evidence, and who can terminate or transition the arrangement. A pod without these decisions simply moves ambiguity into a contract.

Use the triage session to produce three outputs: a ranked debt register, a 90-day repair sequence, and a list of items deliberately left alone. If the team cannot explain why an item is being deferred, it is not triage; it is avoidance.

Related:AI-driven quality assurance and testing services, useful when unreliable regression coverage is the reason every release becomes a manual risk review.

Choose the right debt disposition

Prioritise business exposure, not code quality alone.

Refactoring inside sprints instead of a rewrite project

A rewrite is the wrong default for technical debt because it removes the system’s visible code while preserving its invisible rules. Refactoring inside normal delivery is safer when the product must continue trading, serving customers, or meeting regulated obligations.

The symptom is familiar. A team proposes a replacement platform because the current one is “too messy.” The business approves discovery. Feature work slows while the replacement team models the known requirements. Six months later, the new platform handles the happy path but not the site exceptions, reconciliation steps, permissions, reporting quirks, or recovery procedures that accumulated around the old system.

We use a boundary-first sequence:

1. Choose one business flow. Select a flow with a measurable failure or delay, such as order allocation, payment reconciliation, onboarding, or inventory synchronisation.
2. Capture the current contract. Record inputs, outputs, timing, permissions, failure states, manual steps, and downstream consumers before changing implementation.
3. Add a safety net. Create characterization tests, contract tests, observability, and a rollback path. The first goal is not elegant code; it is making behaviour visible.
4. Place a seam. Introduce an adapter, façade, event boundary, or modular interface that lets the team change one part without rewriting every caller.
5. Move one decision at a time. Route a controlled slice to the new implementation, compare outcomes, and retain the old path until evidence supports removal.
6. Delete the old path. A migration is incomplete while both implementations remain operationally supported without a retirement date.

This is the practical form of the strangler approach, but the name matters less than the discipline: each increment must reduce a known failure mode or make the next change cheaper. A refactor that improves internal style while leaving the same incident pattern untouched is not repayment; it is decoration.

Reserve capacity explicitly. If all sprint capacity is promised to new features, debt work will be displaced by the first incident. If all capacity is assigned to debt, the business will route around engineering and create new uncontrolled workarounds. We set a fixed repair lane, attach each item to a product or operational outcome, and review the lane using the same evidence as feature delivery.

The delivery model affects whether this holds. Staff augmentation supplies people, but the client still has to coordinate the technical lead, QA, DevOps, prioritisation, and handover. Body shopping can increase the number of tickets closed while leaving the failure boundary untouched. A dedicated pod with a tech lead, QA, and DevOps capability can own the outcome across discovery, implementation, validation, deployment, and operational follow-through.

That is the position OmniStack takes: the team should remain on the account long enough to carry context from the first diagnosis through the retirement of the workaround. The pod model is not a substitute for client product authority or regulatory accountability. It is a way to keep delivery risk with a stable team rather than with the client’s hiring pipeline.

Quality gates that stop debt from returning

Technical debt returns when the organisation measures feature completion but does not measure the conditions that made the debt necessary. Quality gates stop recurrence by making contracts, ownership, test evidence, observability, and recovery part of the definition of shipped work.

The root cause is usually not that engineers do not care. It is that the system rewards the local decision: close the ticket, meet the launch date, suppress the alert, or avoid touching the fragile module. The table below maps the failure we see to the gate that prevents its return.

Quality gates must stay proportionate. A low-risk internal report does not need the same release evidence as a payment workflow. A regulated financial service cannot treat recovery, access, auditability, or third-party dependency management as optional engineering polish. Under MAS TRM and APRA CPS 230 contexts, the institution must retain the ability to demonstrate control over technology and operational risk even when delivery involves an external party.

We also separate “green” from “healthy.” A green deployment can still be unhealthy if the new code increased manual reconciliation, weakened traceability, or made the next change dependent on one person. Add operational measures to the release review: incident recurrence, support contacts, manual intervention, recovery time, and the number of new exceptions created.

Documentation is a gate only when it changes execution. A page nobody uses is storage, not control. Link the runbook to the alert, link the contract to the test, link the owner to the service catalogue, and rehearse the recovery path. The 2026 research on large agile organisations is clear on the broader point: communication, collaboration, cooperation, documentation, and requirements are part of the debt system.

When adding a dedicated team is the wrong answer

A dedicated engineering pod will not rescue a business that has no decision owner, no product priority, or no willingness to retire unsafe workarounds. Adding capacity is the wrong choice when the real constraint is unresolved governance.

Hire in house when the work requires a permanent internal authority who will own the domain for years, make trade-offs across departments, and build a capability that remains central after the current debt is repaid. This is particularly true for a core platform architect, a security leader with enterprise-wide mandate, or a product executive accountable for a regulated service.

Do not use a pod to conceal any of the following:

• A roadmap that changes weekly without a decision-maker who can resolve conflicts.
• Production access that cannot be granted or audited.
• Acceptance criteria that are missing because the business process itself is disputed.
• A procurement decision that treats engineers as interchangeable hours.
• A compliance programme that expects the delivery team to assume institutional accountability.

The model also fails when the client expects a temporary group to absorb years of undocumented knowledge without access to the people who hold it. A stable team can investigate, but it cannot infer every exception from source code. Operations, finance, risk, and site leaders must participate in mapping the real process.

Where the constraint is delivery capacity rather than authority, a dedicated pod can be the right tool. The distinction is explicit:

For APAC businesses expanding across locations, the decision often turns on continuity. The same engineers need to understand the site rules, integration edges, release history, and operational consequences. A rotating body supply may fill a sprint but increase the knowledge debt that caused the firefighting in the first place.

If the immediate need is to assess the estate, prioritise debt, and define an accountable repair sequence, request a technical debt assessment with the systems, incident history, roadmap, and control obligations in the room. The useful output is not a generic architecture report. It is a decision about the first boundary to repair, the capacity required, and the owner who will accept the result.

Capacity versus delivery ownership

A pod fits delivery constraints, not unresolved governance or accountability.

The decision to make in the next 30 days

Do not approve a rewrite, open a batch of vague refactoring tickets, or recruit against an undefined “engineering capacity gap.” In the next 30 days, establish whether firefighting is a symptom of one repairable boundary or evidence of a wider operating-model failure.

1. Week one: collect four weeks of engineering work, incidents, releases, manual interventions, and escalations. Calculate the firefighting ratio using actual hours where possible.
2. Week two: map the five highest-impact debt items to affected business processes, sites, controls, owners, and blocked roadmap changes.
3. Week three: select one repairable flow and define its contract, safety net, seam, rollback path, and success evidence.
4. Week four: decide whether the work belongs to the current team, a new internal hire, or a delivery-owning pod. Record why, including the condition that would make the decision wrong.

Set a hard review date for the first repair. If firefighting hours do not fall, recurrence does not change, or the roadmap remains blocked, stop funding activity that only produces patches. Escalate the decision to architecture, product, operations, and risk leadership.

The next decision is specific: name the debt item whose removal will release the most constrained business capacity, name the person accountable for accepting the risk, and choose the team model that can carry it from diagnosis to retirement. Everything else is backlog theatre.

Narrowing the repair focus

1. Four weeks of evidence — Collect work records, incidents, releases, manual interventions, and escalations
2. Five highest-impact debt items — Map affected processes, sites, controls, owners, and blocked changes
3. One repairable flow — Define its contract, safety net, seam, rollback path, and success evidence

FAQ

What is the firefighting ratio in engineering?

The firefighting ratio is unplanned operational engineering work divided by total engineering hours. Include incidents, emergency fixes, production investigation, urgent data correction, and repeated escalations; keep planned reliability investment separate.

How should a company calculate technical debt cost?

Calculate technical debt cost from observed lost capacity and risk: incident response hours, displaced roadmap work, manual reconciliation, delayed operational changes, repeated defects, control remediation, and the exposure created by unresolved failure modes. Avoid unsupported revenue estimates when operational evidence is available.

Should we rewrite a legacy system to remove technical debt?

A rewrite should not be the default. Refactor inside live delivery when the system must continue operating, using a defined business flow, characterization tests, an explicit seam, controlled migration, and a retirement plan for the old path. A rewrite becomes more defensible only when the boundary, replacement scope, migration evidence, and business case are clear.

When should we hire in house instead of using a dedicated engineering pod?

Hire in house when the role requires permanent domain authority, long-term institutional ownership, or executive-level technical decisions that remain central after the debt programme ends. A dedicated pod fits a constrained roadmap when the client can provide product authority and access but needs stable delivery capability across engineering, QA, DevOps, or UX.

Who is accountable when an external team works on regulated financial software?

The regulated institution remains accountable for its technology and operational risk obligations. Under MAS TRM and APRA CPS 230 contexts, the organisation needs clear ownership for service risk, access, incident response, continuity, third-party oversight, evidence, and residual-risk acceptance. An external team can own execution without replacing the institution’s accountability.