💰 Cost and ROI

Cost and ROI: Measuring an automation suite's cost only by its writing time is like measuring a car's cost only by its purchase price — you forget ONGOING costs like fuel, mainte

Measuring an automation suite's cost only by its writing time is like measuring a car's cost only by its purchase price — you forget ONGOING costs like fuel, maintenance and insurance. So why do teams usually make this mistake? Because writing cost is VISIBLE and one-time (it wraps up in a sprint), while maintenance cost is SCATTERED and ongoing (a small piece every sprint). In Java this resembles realizing that integrating a library is easy, but keeping it up to date and resolving dependency conflicts is a continuous job. In QA engineering, the most expensive form of this is the "flaky test": once a test starts randomly going red, the team LEARNS to ignore it instead of fixing it — and that learned distrust eventually leads to a real bug being ignored too.

Maintenance (ongoing)

🎬 The Trust-Erosion Story of a "Flaky" Test

A REAL Bug Slipped Through

A test goes red. The team looks, finds nothing wrong in the code, says "run it again".

On re-run it goes green — as if nothing happened.

This cycle repeats a few more times: red, re-run, green. The test gets labeled "flaky".

The team no longer trusts this test's red result — they LEARN to shrug it off as "probably flaky again".

And one day, that red IS a real bug — but the team has stopped looking, so it slips into production. This is the real cost of flakiness: lost trust.

Step by Step: The Real Cost Items of Automation

One-time: the time spent writing the test, finding locators, adding assertions.

Ongoing: updating locators as the UI changes, fixing broken tests — usually LARGER than the writing cost.

CI minutes, number of parallel workers — every run of a large suite costs real time and real money.

The least visible but most expensive: a flaky test erodes the team's trust in a RED result — this is how real bugs slip through.